-- Logs begin at Wed 2022-08-31 10:22:22 UTC, end at Wed 2022-08-31 11:24:06 UTC. -- Aug 31 10:50:45.762944 dv-cinder-855350-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 10:50:45 2022] *** Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: nodename: dv-cinder-855350-1-iscsi Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: machine: x86_64 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: clock source: unix Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: pcre jit disabled Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: detected number of CPU cores: 4 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: current working directory: / Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: detected binary path: /usr/bin/uwsgi-core Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: your processes number limit is 47680 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: your memory page size is 4096 bytes Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: detected max file descriptor number: 2048 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: lock engine: pthread robust mutexes Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: thunder lock: enabled Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 10:50:45.776047 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 10:50:45.826537 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: Python main interpreter initialized at 0x556ed421c9c0 Aug 31 10:50:45.826537 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: python threads support enabled Aug 31 10:50:45.826537 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: your server socket listen backlog is limited to 100 connections Aug 31 10:50:45.826537 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: your mercy for graceful operations on workers is 90 seconds Aug 31 10:50:45.826946 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 10:50:45.826946 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: *** Operational MODE: preforking *** Aug 31 10:50:45.827260 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: *** uWSGI is running in multiple interpreter mode *** Aug 31 10:50:45.827260 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: spawned uWSGI master process (pid: 81444) Aug 31 10:50:45.827481 dv-cinder-855350-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 10:50:45.828282 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: spawned uWSGI worker 1 (pid: 81462, cores: 1) Aug 31 10:50:45.832036 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: spawned uWSGI worker 2 (pid: 81463, cores: 1) Aug 31 10:50:45.832036 dv-cinder-855350-1-iscsi devstack@keystone.service[81444]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 10:50:50.229792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:50:50.229792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: from cryptography.utils import int_from_bytes Aug 31 10:50:50.230586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:50:50.230586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: from cryptography.utils import int_from_bytes Aug 31 10:50:50.236708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:50:50.236708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: from cryptography.utils import int_from_bytes Aug 31 10:50:50.237165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 10:50:50.237165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: from cryptography.utils import int_from_bytes Aug 31 10:50:50.280584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 10:50:50.280810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 10:50:50.280949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] command line args: [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 10:50:50.281225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 10:50:50.281403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ================================================================================ {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 10:50:50.281949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] admin_token = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.282211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] config_dir = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.282402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.282587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] config_source = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.282769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] debug = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.282975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_config_append = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_dir = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.284970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_file = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.285211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.285462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.285713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.285968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.286149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.286241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 10:50:50.286319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.286418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 10:50:50.287151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] command line args: [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 10:50:50.287241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.287553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 10:50:50.287639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.287814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.287888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ================================================================================ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 10:50:50.288079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] admin_token = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_dir = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.288938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] config_source = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] debug = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.289919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] publish_errors = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_config_append = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.290918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_dir = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] standard_threads = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.291981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] use_journal = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] use_json = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] use_stderr = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.292938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] use_syslog = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.293146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.293257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.293380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.293494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.293610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.293741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.293862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.293979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.294098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.294098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.294306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.294418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.294538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.294658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.294769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.294769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.294976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.295088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.295088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.295351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.20/identity {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.295351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] publish_errors = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.295567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.295686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.295805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.295917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.296043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.296168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.296282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.296401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.296515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.296633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.296751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.296875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] standard_threads = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.296988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.297112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.297225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.297337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.297448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.297561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.297561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_journal = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.297785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.297903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_json = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.298016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.298127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_stderr = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.298245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.298363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] use_syslog = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.298363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 10:50:50.298588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.298714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.298919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.299957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.300884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.301922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.302042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.302978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.303914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.304156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.304156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.304412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.304525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.304637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.305740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.305864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.306960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.306960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.307732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.307732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.307732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.308116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.308116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.308116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.309432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.309559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.309676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.309676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.309897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.310984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.311931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.312908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.313934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.314898 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.315944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.316989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.317971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.318896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.319923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.320964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.321897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.322923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.external = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.323957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.password = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] auth.token = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.324931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.325889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.326956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.327886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.external = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.328957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.password = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] auth.token = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.329982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.330918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.331967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.332896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.333950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.334879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335115 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.335971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.336934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.337980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.338999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.339936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340207 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.340904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.341949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.342983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.343909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.344977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.345347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.345463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.345577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.345699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.345815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.345929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346510 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.346973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.347902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.348855 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.349502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.349621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.349745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.349860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.349976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.350897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.351903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.352032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.352976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.353915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.354959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.355997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.356988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] role.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] role.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.357912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.358985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.359992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.360834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.361979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.362910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.363953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.364124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.364240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.364353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.364466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.364634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.365963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.366993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.367910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.368974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.369916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.connection = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.370967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.371887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.372971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.373895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81462) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 10:50:50.374462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.374918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.375142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.375388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.375654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.375905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.376184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.376429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.376675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.376915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.377156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.377395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.377636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.377886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.378129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.378377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.378624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.378875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.379114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.379362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.379599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.379843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.380106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 10:50:50.380396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81463) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 10:50:50.403524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.403759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.403990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.405805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.406040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.406269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.413801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.415828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.442794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.443085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.444974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.445210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.498580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.498740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.548059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.548296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.549571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.549819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.704456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.704679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.704902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.704989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.705784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.706006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.730130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.730375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.730594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.730720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.730805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.730960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.731860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.732189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.742623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.743593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.755510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.755705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.755852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.755948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.756982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81462) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.757083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.757284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81463) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 10:50:50.774298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.774512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:50:50.774750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.774927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.775042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:50:50.775162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:50:50.775299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.775408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.775509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 10:50:50.775621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.775769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.775879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.776063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.776174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.776334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.776866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.785701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.785988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.786265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.786361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.787119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.787405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.787570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.788030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.794454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.795044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.797409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.797702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.797993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.798083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.798283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.798362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.798568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.798648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.798834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.798926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.799933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.800036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.800233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.800354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.800639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.800922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.822947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.824088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.824442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.824732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.825563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.825853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.829321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.829614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.829900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.830531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.830828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.831111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.836978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.837270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.838319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.838614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.841269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.841563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.841850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.842138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.842417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.842839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.843158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.843449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.843792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.844113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.844397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.844476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.844682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.844773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.844965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.845296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.861305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.861589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.864283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.864824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.865751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.868776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.868930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.869234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.871992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.872314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.873616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.876545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.876793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.879699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.879908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.880776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.881065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.881353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.881636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.882021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.883758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.884096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.884386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.884673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.884957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.885241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.898882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.899174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.899445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.899710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.901894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.902192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.902466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.902737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.906500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.908209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.909663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.909783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.910065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.911220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.912858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.913124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.913214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.913423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.913505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.913711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.914015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.916105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.916406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.916805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.917105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.917398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.925316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.925608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.925895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.926179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.926467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.926760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.927078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.928900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.929191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.929578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.929868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.930157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.930452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.930747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.942514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.946179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.947116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.949122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.950075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.950589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.952099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.952398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.952692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.952999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.954445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.956107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.956399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.956687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.960069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.961566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.964073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.964488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.964786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.965080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.965366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.965573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.968623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.968926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.969220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.969503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.973585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.973886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.974803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.978496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.978796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.979081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.981899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81462) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.982206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.982494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.982774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.983053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.983328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.983618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.983900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.984207 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.984482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.984757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.985032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.985301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.985578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81462) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.986059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81463) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 10:50:50.986347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.986628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.986904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.987453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.987728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.988025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.988319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.988595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.988869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.989141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.989416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:50.990294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81463) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 10:50:51.026703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556ed421c9c0 pid: 81463 (default app) Aug 31 10:50:51.029962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x556ed421c9c0 pid: 81462 (default app) Aug 31 10:50:51.030202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9ba1b-7e44-44b2-b210-1f27520bb8ad None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:50:51.030496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9ba1b-7e44-44b2-b210-1f27520bb8ad None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:50:51.030776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe9ba1b-7e44-44b2-b210-1f27520bb8ad None None] PATH_INFO: `/v3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:50:51.034967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1/1] 10.222.0.20 () {50 vars in 791 bytes} [Wed Aug 31 10:50:51 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:50:57.653421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:50:57.653935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:50:57.653935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:50:57.658534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1/2] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:50:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:50:57.663835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:50:57.664421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:50:57.664421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:50:57.735109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: INFO dbcounter [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] Registered counter for database keystone Aug 31 10:50:57.754850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG oslo_db.sqlalchemy.engines [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81463) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 10:50:57.761614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writer thread running {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 10:50:57.945043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG passlib.handlers.bcrypt [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81463) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 10:50:57.945502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG passlib.handlers.bcrypt [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81463) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 10:50:58.005000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:50:58.006608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:50:58.136859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 2/3] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:50:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 10:50:58.551181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:50:58.620541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: INFO dbcounter [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] Registered counter for database keystone Aug 31 10:50:58.642387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG oslo_db.sqlalchemy.engines [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81462) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 10:50:58.649571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writer thread running {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 10:50:58.773036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2f507-abd9-4f27-98d4-7bb19ec6f023 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:50:58.777649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:50:58.778285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:50:58.778566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:50:58.778840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:50:58.787557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:50:58.839084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING keystone.common.rbac_enforcer.enforcer [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 10:50:58.839351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00bbf9-3de9-4416-b65a-c1e6fb39de4c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:50:58.849849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 2/4] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Aug 31 10:50:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 10:50:58.855812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:50:58.870911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d85a2-d543-4830-bb2d-8607f2f96a54 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:50:58.875424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:50:58.876565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:50:58.876851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:50:58.877124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] PATH_INFO: `/v3/projects/34b2b6cdb81943ad9b9237874eeca93b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:50:58.878716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] RBAC: Authorizing `identity:get_project(project_id=34b2b6cdb81943ad9b9237874eeca93b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:50:58.933524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 10:50:58.933872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7df37a-eb89-45d0-942b-8025e02788da admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:50:58.938452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 3/5] 10.222.0.20 () {56 vars in 1207 bytes} [Wed Aug 31 10:50:58 2022] GET /identity/v3/projects/34b2b6cdb81943ad9b9237874eeca93b => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:00.822935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebdcce-a741-476f-b562-8151ad5d52f3 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:00.823415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebdcce-a741-476f-b562-8151ad5d52f3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:00.823524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebdcce-a741-476f-b562-8151ad5d52f3 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:00.827584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 3/6] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:00.833486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:00.834018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:00.834018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:00.855489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:00.857059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:00.865446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 4/7] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 10:51:01.133440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebdcce-a741-476f-b562-8151ad5d52f3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:01.145817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebdcce-a741-476f-b562-8151ad5d52f3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:01.150041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681f62a-1243-4d5d-9c6d-afa21a66f98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:01.150641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681f62a-1243-4d5d-9c6d-afa21a66f98a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:01.150923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681f62a-1243-4d5d-9c6d-afa21a66f98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:01.151198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681f62a-1243-4d5d-9c6d-afa21a66f98a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:01.159002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681f62a-1243-4d5d-9c6d-afa21a66f98a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:01.160867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681f62a-1243-4d5d-9c6d-afa21a66f98a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:01.170110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 4/8] 10.222.0.20 () {58 vars in 1324 bytes} [Wed Aug 31 10:51:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 10:51:01.175801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:01.185361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b17d9f-5c69-49e6-b66f-08d87b7accfe None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:01.189753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43e92b-0369-4771-91dc-cc676491b46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:01.190380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43e92b-0369-4771-91dc-cc676491b46f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:01.190665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43e92b-0369-4771-91dc-cc676491b46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:01.190941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43e92b-0369-4771-91dc-cc676491b46f admin admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:01.192422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43e92b-0369-4771-91dc-cc676491b46f admin admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:01.194417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43e92b-0369-4771-91dc-cc676491b46f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:01.195969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 5/9] 10.222.0.20 () {56 vars in 1198 bytes} [Wed Aug 31 10:51:01 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:03.164926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fbd8c-50cd-4fed-9df6-1f5124cd280f None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.165475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fbd8c-50cd-4fed-9df6-1f5124cd280f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.165475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fbd8c-50cd-4fed-9df6-1f5124cd280f None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.166791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 5/10] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:03.171807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.172281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.172510 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.191132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:03.192949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:03.221200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644914-96ec-412b-a044-a489fe2da54c None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.221505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644914-96ec-412b-a044-a489fe2da54c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.221783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f644914-96ec-412b-a044-a489fe2da54c None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.223124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 6/11] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:03.228768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.229074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.229352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.273019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG passlib.handlers.bcrypt [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81462) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 10:51:03.273416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG passlib.handlers.bcrypt [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81462) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 10:51:03.279657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 6/12] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:03.323324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:03.324956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:03.428376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 7/13] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:03.549074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:03.572811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:03.655524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4876db-e3b7-4998-884e-a59c8d49b659 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:03.690793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:03.691364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.691650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.692458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.704647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:03.709050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:03.714571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:03.715744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:03.716836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 7/14] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:03.722415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:03.810522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:03.820230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8793beb-c1d7-4ccd-8a43-c4be957a551a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:03.824660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:03.830756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbcfc2e-6e09-44f3-92fa-d85d10a6f0b0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:03.863476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:03.864444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.864725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.864995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.866580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:03.867221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:03.867554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:03.867840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:03.868550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:03.868853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:03.870262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:03.883347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 8/15] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:03.888918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:03.933081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:03.934597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:03.987786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd4b3e-909e-4642-88a5-325f46cd5266 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:04.023335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:04.023977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:04.024290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:04.024571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:04.025732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:04.027190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:04.028812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 9/16] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:03 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:04.033670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 8/17] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:51:04.034458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:04.039017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:04.041829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:04.049886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:04.137235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd729d-2951-429e-95ad-25f7d858ceef None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:04.137973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e725f2a-0546-4cdf-80d9-0477017f2a93 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:04.171629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:04.172526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:04.172809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:04.173081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:04.179842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:04.180507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:04.180787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:04.181061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:04.182191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:04.183648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:04.185275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 10/18] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:04.191069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:04.194290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:04.195787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:51:04.214610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:04.215565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:51:04.216398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:51:04.216398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:51:04.216398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:04.216398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 9/19] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:04.221866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:04.283174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:04.283353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:04.305263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d36123-abb0-4960-bce5-c835110079eb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:04.331465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6657e-f179-456d-bd9f-3fcebc924dc0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:04.341418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:04.342311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:04.342617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:04.342899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:04.343626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.domain.id=default, user.id=a29c05a14dda47f2a0f073398ac0dadb) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:04.343934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:04.345684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4687028-ed9d-45dd-8a5e-33330325c430 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:04.353126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 11/20] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Aug 31 10:51:04 2022] GET /identity/v3/role_assignments?user.id=a29c05a14dda47f2a0f073398ac0dadb&scope.domain.id=default&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:04.368189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:04.368823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:04.369096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:04.369365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:04.370032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:04.370354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:04.371800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44caf656-0934-4839-b273-3ce74711c2df None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:04.392278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 10/21] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:06.569059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b6df4-5d41-4e81-be4b-d6fbfa7d820a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:06.569583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b6df4-5d41-4e81-be4b-d6fbfa7d820a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:06.570062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b6df4-5d41-4e81-be4b-d6fbfa7d820a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:06.572544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 12/22] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:06.578491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20502383-738a-48bd-b513-13007b5f3121 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:06.579351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20502383-738a-48bd-b513-13007b5f3121 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:06.579801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20502383-738a-48bd-b513-13007b5f3121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:06.609023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-20502383-738a-48bd-b513-13007b5f3121 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:06.610449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-20502383-738a-48bd-b513-13007b5f3121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:06.695744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 11/23] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:06.714684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c65e1-a28b-42af-b3f4-ce343b356b88 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:06.715228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c65e1-a28b-42af-b3f4-ce343b356b88 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:06.715654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c65e1-a28b-42af-b3f4-ce343b356b88 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:06.717522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 13/24] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:06.723061 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:06.723345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:06.723625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:06.754031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:06.756022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:06.791671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8281f-a765-452c-b72c-e568928e7f67 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:06.792336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8281f-a765-452c-b72c-e568928e7f67 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:06.792823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8281f-a765-452c-b72c-e568928e7f67 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:06.794511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 14/25] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:06.800110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:06.800665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:06.801224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:06.847542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:06.849133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:06.853530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:06.906733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 12/26] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:06.974639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 15/27] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:07.179862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:07.308515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b075fb0f-93b5-479d-bc3e-53030a2f88c5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:07.324129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:07.396819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:07.397534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:07.397823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:07.398103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:07.426461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:07.433487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:07.434509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:07.435482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:07.437399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 13/28] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:07.438297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:07.480958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:07.484356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:07.555138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b064a41-78a2-4daf-8d06-a340ecd4a743 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:07.559462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05902e98-69c7-44c6-a406-62e78508bcea None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:07.600935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:07.601554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:07.601815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:07.602087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:07.602765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:07.604187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:07.619697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:07.620359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:07.620636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:07.620910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:07.656886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:07.658339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:51:07.683393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:07.684386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:51:07.685299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:51:07.685299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:51:07.685299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:07.685299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 16/29] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:07.685299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:07.793512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 14/30] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 10:51:07 2022] POST /identity/v3/roles => generated 219 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:07.794562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:07.902814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede8251-3353-4b01-9610-4263b07ddf01 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:07.933933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:07.945840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:07.949725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:07.950612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:07.951046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:07.951460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:07.952465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:07.952933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:07.955517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b8e55d-965c-4e71-a0b0-3a4011dc6572 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:07.964766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be135cd-d6cc-4e63-aa02-f9ff3a400ed1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:07.968979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 17/31] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:07.971688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d84220-aa2c-4afa-a2c2-3996cfb3ccf2 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:07.972190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d84220-aa2c-4afa-a2c2-3996cfb3ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:07.972664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d84220-aa2c-4afa-a2c2-3996cfb3ccf2 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:07.975360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 18/32] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:07.976300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d84220-aa2c-4afa-a2c2-3996cfb3ccf2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:08.049569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:08.050216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.050506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.050785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.051758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:08.052315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:08.053899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21b739-d9f0-41dd-9c56-2ab75fce5e6a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:08.069084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 15/33] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:08.070497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.070796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.071078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.096353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:08.098630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:08.131800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d84220-aa2c-4afa-a2c2-3996cfb3ccf2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:08.179144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:08.179842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.180194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.180475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.181760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:08.183397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:08.185228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 19/34] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:07 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:08.186100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:08.196233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 16/35] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:08.197977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:08.267100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:08.312907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:08.316506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d36f-6282-43d1-a69d-dd0dfc1369bd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:08.339087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4cd27-1570-4495-9dac-3c37fae01595 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:08.355462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c51f1df-a374-4700-8877-4e7df63a049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:08.356102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f1df-a374-4700-8877-4e7df63a049c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.356384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f1df-a374-4700-8877-4e7df63a049c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.356737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c51f1df-a374-4700-8877-4e7df63a049c None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.357375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51f1df-a374-4700-8877-4e7df63a049c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:08.358872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c51f1df-a374-4700-8877-4e7df63a049c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:08.383942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5875d8-4efb-4ce6-a209-cd40338c487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:08.385504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5875d8-4efb-4ce6-a209-cd40338c487c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.386052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5875d8-4efb-4ce6-a209-cd40338c487c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.386556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5875d8-4efb-4ce6-a209-cd40338c487c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.389423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5875d8-4efb-4ce6-a209-cd40338c487c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:08.392449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5875d8-4efb-4ce6-a209-cd40338c487c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:08.394821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 17/36] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:08.397039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a93fc5-e3eb-4dfd-aa53-589839096e9d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.397546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a93fc5-e3eb-4dfd-aa53-589839096e9d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.398090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a93fc5-e3eb-4dfd-aa53-589839096e9d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.399615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 18/37] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:08.406715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a93fc5-e3eb-4dfd-aa53-589839096e9d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:08.515774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a93fc5-e3eb-4dfd-aa53-589839096e9d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:08.516947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 20/38] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:51:08 2022] POST /identity/v3/projects => generated 301 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:08.518622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6aadf-de7b-4dc4-8f81-eada5180f497 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.518919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6aadf-de7b-4dc4-8f81-eada5180f497 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.519202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6aadf-de7b-4dc4-8f81-eada5180f497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.547069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e7c6aadf-de7b-4dc4-8f81-eada5180f497 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:08.548832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7c6aadf-de7b-4dc4-8f81-eada5180f497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:08.556438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:08.557123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.557407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.557688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] PATH_INFO: `/v3/domains/default/users/a29c05a14dda47f2a0f073398ac0dadb/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.559957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a29c05a14dda47f2a0f073398ac0dadb, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:08.561483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:08.584798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 19/39] 10.222.0.20 () {58 vars in 1361 bytes} [Wed Aug 31 10:51:08 2022] PUT /identity/v3/domains/default/users/a29c05a14dda47f2a0f073398ac0dadb/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:08.586045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:08.608053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:08.667339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 21/40] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:08.669553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7b82e-d7f5-4250-aaac-cd2bda106814 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.670063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7b82e-d7f5-4250-aaac-cd2bda106814 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.670592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7b82e-d7f5-4250-aaac-cd2bda106814 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.672285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 22/41] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:08.678693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.680049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.680462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.706460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:08.707911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:08.744280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3592b7b-4afe-42ef-a3f4-42ff9cbae3bf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:08.755279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:08.805555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:08.806205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:08.806491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:08.806776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:08.807966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:08.811062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:08.821592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 23/42] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:08.832484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 20/43] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 10:51:08 2022] POST /identity/v3/roles => generated 225 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:08.985030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:09.101144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38676401-7544-4ebf-9873-5174d87348a9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:09.108793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:09.142131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:09.142856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:09.143139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:09.143418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:09.144152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:09.146256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:09.167987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 24/44] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 10:51:08 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:09.204203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:09.237544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fdf96-e39a-4e85-9804-75cb5e4ff9a3 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:09.277185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1e864-902b-41a5-b3c1-817572f04bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:09.277918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1e864-902b-41a5-b3c1-817572f04bbb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:09.278199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1e864-902b-41a5-b3c1-817572f04bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:09.278516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1e864-902b-41a5-b3c1-817572f04bbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:09.279733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1e864-902b-41a5-b3c1-817572f04bbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:09.281377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1e864-902b-41a5-b3c1-817572f04bbb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:09.283433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 21/45] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:09.289381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:09.427659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:09.505250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6561c-7876-4f4a-b187-23aa8cf9b833 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:09.539127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd1dfd-dff7-44d5-8f30-9d5ca300303e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:09.539745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd1dfd-dff7-44d5-8f30-9d5ca300303e None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:09.540052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd1dfd-dff7-44d5-8f30-9d5ca300303e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:09.540331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd1dfd-dff7-44d5-8f30-9d5ca300303e None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:09.541003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd1dfd-dff7-44d5-8f30-9d5ca300303e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:09.542454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd1dfd-dff7-44d5-8f30-9d5ca300303e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:09.569965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 25/46] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:51:09 2022] POST /identity/v3/projects => generated 312 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:51:10.595302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dfb12-d159-46bf-bdce-a1d0e2a28131 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:10.595818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dfb12-d159-46bf-bdce-a1d0e2a28131 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:10.595926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dfb12-d159-46bf-bdce-a1d0e2a28131 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:10.597285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 22/47] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:10.603033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:10.603493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:10.603580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:10.621673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:10.622881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:10.692134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 26/48] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:10.964344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dfb12-d159-46bf-bdce-a1d0e2a28131 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:11.067873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dfb12-d159-46bf-bdce-a1d0e2a28131 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:11.102032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:11.102674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.102961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.103242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.114818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:11.119717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:11.120712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:11.121716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 23/49] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:11.126695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:11.140211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:11.231147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b14c0-75c4-41c7-9fd1-d7edda244ef2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:11.266041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:11.266658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.266943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.267219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.267889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:11.268224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:11.269638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:11.278075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 27/50] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:11.283641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:11.287190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:11.385366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a70e51-d18a-47ef-9916-d3afb16f750d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:11.419240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102244f-8343-47d8-a18a-6891da5a7216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:11.419875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102244f-8343-47d8-a18a-6891da5a7216 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.420193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102244f-8343-47d8-a18a-6891da5a7216 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.420471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102244f-8343-47d8-a18a-6891da5a7216 None admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.421605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102244f-8343-47d8-a18a-6891da5a7216 None admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:11.423059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102244f-8343-47d8-a18a-6891da5a7216 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:11.424688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 24/51] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:11 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:11.431131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:11.497040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2810-4a1f-45e9-bd46-5aad2c3e63ba None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.497240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2810-4a1f-45e9-bd46-5aad2c3e63ba None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.497524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe2810-4a1f-45e9-bd46-5aad2c3e63ba None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.498865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 25/52] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:11.503803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.504748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.505043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.506761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:11.524708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:11.526005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:11.537339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5f22e-df09-4bdb-8c87-2f5d0d3245a4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:11.572454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:11.573109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.573389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.573661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.574844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:11.576370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:11.578030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 28/53] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:11.584463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:11.690036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba239cad-fc85-488b-827c-14b5e9908fee None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:11.696457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:11.701688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 26/54] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:11.725175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:11.725785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:11.726072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:11.726349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:11.727075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.domain.id=default, user.id=a29c05a14dda47f2a0f073398ac0dadb) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:11.727385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:11.728877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:11.734559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 29/55] 10.222.0.20 () {56 vars in 1322 bytes} [Wed Aug 31 10:51:11 2022] GET /identity/v3/role_assignments?user.id=a29c05a14dda47f2a0f073398ac0dadb&scope.domain.id=default&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:11.972453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:12.069958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bcd126-e5d8-454a-80fb-6315a6de7b5a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:12.105163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d302e1b9-daa4-476c-a9ad-94c7472eda77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:12.105799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302e1b9-daa4-476c-a9ad-94c7472eda77 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:12.106083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302e1b9-daa4-476c-a9ad-94c7472eda77 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:12.106356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d302e1b9-daa4-476c-a9ad-94c7472eda77 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:12.107468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302e1b9-daa4-476c-a9ad-94c7472eda77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:12.108960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d302e1b9-daa4-476c-a9ad-94c7472eda77 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:12.110557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 27/56] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:12.116424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:12.120348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:12.217102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f69fe-e05b-47d3-812c-4ed6a7ad019d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:12.249875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62df00-87e4-4de9-a034-91c3dde131fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:12.250478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62df00-87e4-4de9-a034-91c3dde131fe None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:12.250759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62df00-87e4-4de9-a034-91c3dde131fe None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:12.251035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62df00-87e4-4de9-a034-91c3dde131fe None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:12.251701 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62df00-87e4-4de9-a034-91c3dde131fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:12.253187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62df00-87e4-4de9-a034-91c3dde131fe None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:12.280106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 30/57] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Aug 31 10:51:12 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:14.203259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b422c2-146c-47c1-8dec-9abbb8ede49a None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:14.203778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b422c2-146c-47c1-8dec-9abbb8ede49a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:14.203864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b422c2-146c-47c1-8dec-9abbb8ede49a None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:14.205292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 28/58] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:14.211369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:14.211918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:14.211918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:14.230898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:14.232166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:14.279657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:14.296454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 31/59] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:14.570496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b422c2-146c-47c1-8dec-9abbb8ede49a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:14.636716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:14.670099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b422c2-146c-47c1-8dec-9abbb8ede49a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:14.703875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235b8f7-6000-4581-a0f9-9cbd309e1e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:14.704559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b8f7-6000-4581-a0f9-9cbd309e1e8b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:14.704843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b8f7-6000-4581-a0f9-9cbd309e1e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:14.705122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235b8f7-6000-4581-a0f9-9cbd309e1e8b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:14.706240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235b8f7-6000-4581-a0f9-9cbd309e1e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:14.708047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235b8f7-6000-4581-a0f9-9cbd309e1e8b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:14.709627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 29/60] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:14.715174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:14.812517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2d87-0f9e-499d-89b6-22615b9bd21b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:14.845261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d902cc-4bd4-44aa-94cf-59f7e10dd4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:14.845793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d902cc-4bd4-44aa-94cf-59f7e10dd4d1 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:14.846075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d902cc-4bd4-44aa-94cf-59f7e10dd4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:14.846347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d902cc-4bd4-44aa-94cf-59f7e10dd4d1 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:14.847076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d902cc-4bd4-44aa-94cf-59f7e10dd4d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:14.849266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d902cc-4bd4-44aa-94cf-59f7e10dd4d1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:14.875660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 32/61] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:14 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:17.875978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172f71-5573-4901-a6e7-4183c3f23ae7 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:17.876518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172f71-5573-4901-a6e7-4183c3f23ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:17.876593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172f71-5573-4901-a6e7-4183c3f23ae7 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:17.877896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 30/62] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:17.883328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e08cb-ba21-4f8f-ae5f-84078cf79fa7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:17.883756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e08cb-ba21-4f8f-ae5f-84078cf79fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:17.883928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e08cb-ba21-4f8f-ae5f-84078cf79fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:17.909071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9a9e08cb-ba21-4f8f-ae5f-84078cf79fa7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:17.910817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a9e08cb-ba21-4f8f-ae5f-84078cf79fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:17.953703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:18.000428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 33/63] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:18.217634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40ff47-2b86-491e-8c2b-357389fb4268 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.218269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40ff47-2b86-491e-8c2b-357389fb4268 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.219013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40ff47-2b86-491e-8c2b-357389fb4268 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.220624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 31/64] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:18.226395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26302ce4-6610-4e59-b8aa-450f8eed6e81 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.226693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26302ce4-6610-4e59-b8aa-450f8eed6e81 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.226978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26302ce4-6610-4e59-b8aa-450f8eed6e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.254440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-26302ce4-6610-4e59-b8aa-450f8eed6e81 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:18.255654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-26302ce4-6610-4e59-b8aa-450f8eed6e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:18.344577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40ff47-2b86-491e-8c2b-357389fb4268 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:18.346653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 34/65] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:18.465733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aac4a8-a361-47b4-a453-ea8044a07e66 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.466382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aac4a8-a361-47b4-a453-ea8044a07e66 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.466878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aac4a8-a361-47b4-a453-ea8044a07e66 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.472990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 35/66] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:18.478561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1ae0-94da-4ae7-b4be-434005c3e83e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.479087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1ae0-94da-4ae7-b4be-434005c3e83e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.479583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b1ae0-94da-4ae7-b4be-434005c3e83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.509543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40ff47-2b86-491e-8c2b-357389fb4268 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:18.514248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e37b1ae0-94da-4ae7-b4be-434005c3e83e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:18.515719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e37b1ae0-94da-4ae7-b4be-434005c3e83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:18.547015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c6314-dea0-424e-bd02-4ae434255fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:18.548587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c6314-dea0-424e-bd02-4ae434255fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.549337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c6314-dea0-424e-bd02-4ae434255fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.549887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c6314-dea0-424e-bd02-4ae434255fe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.553848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:18.567112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c6314-dea0-424e-bd02-4ae434255fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4a2c6314-dea0-424e-bd02-4ae434255fe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:18.573949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:18.575057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:18.576022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:18.577383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 32/67] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:18.578435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133db027-03d0-4ec6-a64b-6ef381e5a7e0 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.578719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133db027-03d0-4ec6-a64b-6ef381e5a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.579306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133db027-03d0-4ec6-a64b-6ef381e5a7e0 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.584435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 33/68] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:18.584435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2ad29-c995-4118-af63-53ac85dd764c None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.584435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2ad29-c995-4118-af63-53ac85dd764c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.584435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2ad29-c995-4118-af63-53ac85dd764c None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.584435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 34/69] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:18.584784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2ad29-c995-4118-af63-53ac85dd764c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:18.602936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 36/70] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:18.605150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841d9bd-4e53-4551-90e2-d3cf83258c90 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.605658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841d9bd-4e53-4551-90e2-d3cf83258c90 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.606127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841d9bd-4e53-4551-90e2-d3cf83258c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.629566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e841d9bd-4e53-4551-90e2-d3cf83258c90 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:18.631707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e841d9bd-4e53-4551-90e2-d3cf83258c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:18.664346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:18.701635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2ad29-c995-4118-af63-53ac85dd764c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:18.718489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 37/71] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:18.720164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.720636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.720918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.740323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:18.741512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:18.747420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:18.748318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.748799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.749316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.750639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:18.751123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:18.753206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:18.769930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 35/72] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:18.771339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:18.826227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 38/73] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:18.827450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:18.850346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:18.927692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7696db3-ddb3-4859-a404-15bd80374b02 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:18.971930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:18.975452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:18.976450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:18.976646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:18.976921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:18.991346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813c6e8b-717f-47ae-b39d-65058090751d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:18.996204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:19.003094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:19.004241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:19.004986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:19.005555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 36/74] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:19.006237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.036803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.037600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.037955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.038304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.051945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.053437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.055094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 39/75] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:18 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.055906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.119750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852f350-3014-48a5-9275-cbc20418a77e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.137632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.157012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.157647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.157957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.158233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.161944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5e7c4-effd-4fbe-bf21-15b884ebd0a8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.168982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:19.173808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:19.174638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:19.175904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 37/76] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:19.181752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.205442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.206075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.206357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.206665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.216797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:19.221911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:19.222890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:19.223650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 40/77] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:19.224385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.240126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.289736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cf7eb-15a8-4aa0-81d5-bc6f2c943874 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.326328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.326935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.327215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.327493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.328194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:19.328501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.329901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.338277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 38/78] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.341273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.344289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a878a333-a91a-44ad-9c89-075e6dbdcf9e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.383390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.384076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.384422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.384744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.385385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.443730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ccbf4-fe21-4e6d-8c8f-22f5b854dc8d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.453713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.455195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.465995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.480087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.480730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.481015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.481298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.482419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.483874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.485493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 39/79] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.486262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.555469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 41/80] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:19.556377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.589166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477d87f-4ce0-4db3-9c12-bce83b6b3a42 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.599081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.624416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.625032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.625316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.625596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.626257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:19.626564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.627957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.637423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 40/81] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.638224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.660269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e0be7c-7ae3-4ee5-b8bc-5fb63a82983f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.671463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.696352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.696962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.697245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.697524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.698190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:19.698499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.699881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.708758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 42/82] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.709310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.748259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01f52cd-e4fb-46c8-a352-cb37dadf343e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.787183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.787922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.788303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.788584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.790100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.792508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.794100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 41/83] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.794874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.820335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3477f-fb5a-439f-8aa2-2bacf9b1f53f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.834525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.856276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.856887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.857170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.857444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.858091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.859537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.881105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 43/84] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:19 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:19.881930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:19.911509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30977f10-1432-4ce1-a224-2b8baa8a95ac None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:19.918924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:19.958314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:19.959069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:19.959350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:19.959650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:19.960930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:19.962420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:19.969571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 42/85] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:19.969571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.026133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d91e4-e610-4b53-8faf-2b203d29ec1d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.071524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.072173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.072465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.072749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.074658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.076688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.078716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 44/86] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:20.079522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.090492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523670d-d85a-466d-bd4c-0acc5f9c94de None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.100881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:20.132936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.132936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.132936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.133270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.136039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.137726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.138344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 43/87] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:19 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:20.139119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.200837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:20.207272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2e8e8-a2e9-460b-8bfd-8ea30a0c38ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.248982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.249777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.250063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.250337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.275513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca6c1ec-7b85-4fd6-89e3-12eb84f0e68d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.313556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.314222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.314507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.314780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.315513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=940a93c8f7514875b6777c4fd2aaa85a) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:20.315824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.317699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.326039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 44/88] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:20 2022] GET /identity/v3/role_assignments?user.id=940a93c8f7514875b6777c4fd2aaa85a&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:20.326792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.351250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.353846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.374637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:20.445540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fce31b-26f2-4b6b-b1a4-df6986c799a6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.492161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.495791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.496044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.496327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.521099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 45/89] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:20.521761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.544379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:20.612119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.614178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.666029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:20.680046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b1d65-7a97-4a03-a417-d9ea64abd2b1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.716531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.717158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.717443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.717725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.804376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:20.812824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 45/90] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:20.813691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.814870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.817039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.931210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b73f8-7b73-4e19-ac33-50989ae6ac24 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:20.968519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:20.970232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:20.970734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:20.971231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:20.973360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:20.975019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:20.977347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 46/91] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:20 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:20.978336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:20.987230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:21.089449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f7351-8c79-496b-85ef-b39f5213204e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:21.099117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 46/92] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:21.099991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:21.126797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:21.127796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:21.128899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:21.129463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] PATH_INFO: `/v3/projects/546c1d0af93e410292a37a557b0b45f6` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:21.130860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] RBAC: Authorizing `identity:get_project(project_id=546c1d0af93e410292a37a557b0b45f6)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:21.132630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:21.134506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 47/93] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:20 2022] GET /identity/v3/projects/546c1d0af93e410292a37a557b0b45f6 => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:21.135487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:21.216297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:21.265675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df39ed-8104-4abb-9a99-4ed281aa66e2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:21.297188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37a392a-19a9-4405-a017-b373ea36da8d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:21.320420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:21.344351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:21.344994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:21.345283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:21.345560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:21.347323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:21.349043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:21.352099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 48/94] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:21 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:21.352649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:21.360542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:21.361167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:21.361451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:21.361729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:21.362470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=a29c05a14dda47f2a0f073398ac0dadb) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:21.363321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:21.368346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:21.380111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 47/95] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:21 2022] GET /identity/v3/role_assignments?user.id=a29c05a14dda47f2a0f073398ac0dadb&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:21.383845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:21.489409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604361b-678b-41f6-a982-4e9fc3dd575c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:21.527329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:21.527991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:21.528310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:21.528588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:21.529322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=546c1d0af93e410292a37a557b0b45f6, user.id=940a93c8f7514875b6777c4fd2aaa85a) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:21.529639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:21.531085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575d858-955e-415f-837f-3bbb29dbf8a4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:21.537865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 49/96] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:21 2022] GET /identity/v3/role_assignments?user.id=940a93c8f7514875b6777c4fd2aaa85a&scope.project.id=546c1d0af93e410292a37a557b0b45f6&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:21.612599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0019edf-05fa-4976-ba47-bac32736ab2e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:21.663738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:21.664400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:21.664687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:21.664967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:21.665739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] RBAC: Adding query filter params (role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=940a93c8f7514875b6777c4fd2aaa85a) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:21.666056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:21.667550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f22644-0af4-48a7-abfb-3595d14f9784 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:21.678260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 48/97] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:21 2022] GET /identity/v3/role_assignments?user.id=940a93c8f7514875b6777c4fd2aaa85a&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:22.309603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bc486-d6ec-4b84-be6b-b07a44923e1f None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:22.310119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bc486-d6ec-4b84-be6b-b07a44923e1f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:22.310193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bc486-d6ec-4b84-be6b-b07a44923e1f None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:22.311514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 50/98] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:22.321819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faf8b-31e2-4b65-996d-32d75899c716 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:22.322285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faf8b-31e2-4b65-996d-32d75899c716 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:22.322358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415faf8b-31e2-4b65-996d-32d75899c716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:22.349111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-415faf8b-31e2-4b65-996d-32d75899c716 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:22.350315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-415faf8b-31e2-4b65-996d-32d75899c716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:22.372797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:22.436465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 49/99] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:22.543037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a473f0-184a-484e-8fe0-30908530f972 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:22.543376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a473f0-184a-484e-8fe0-30908530f972 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:22.543603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a473f0-184a-484e-8fe0-30908530f972 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:22.544994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 51/100] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:22.550243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:22.550541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:22.550820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:22.569618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:22.570724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:22.640979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 50/101] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:22.854437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a473f0-184a-484e-8fe0-30908530f972 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:22.864632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:22.973537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a473f0-184a-484e-8fe0-30908530f972 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:23.010019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:23.010680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.010965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.011242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.024556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:23.029433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:23.030287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:23.031283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 52/102] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:23.037109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:23.155283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:23.171950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:23.191842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b27495-e692-44d4-9a43-7869246c639a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:23.244927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:23.245564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.245851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.246415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.246818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:23.247132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:23.248552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:23.265831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 51/103] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:23.271759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:23.298694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:23.370007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1aa43-3846-4d30-b542-40ed28eefe2c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:23.393157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ccb24c-8c69-4576-9f0a-e28891ec15a7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:23.442059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f6d09-409a-4837-b68b-57bfd9f7eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:23.442626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f6d09-409a-4837-b68b-57bfd9f7eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.442912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f6d09-409a-4837-b68b-57bfd9f7eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.443196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f6d09-409a-4837-b68b-57bfd9f7eb91 None admin] PATH_INFO: `/v3/users/164e98ad65ed4afb9e8c7c6d9fdb53db` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.452497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aa54f8-61e5-4746-a578-485ca0bea4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:23.453139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa54f8-61e5-4746-a578-485ca0bea4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.453426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa54f8-61e5-4746-a578-485ca0bea4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.453706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aa54f8-61e5-4746-a578-485ca0bea4f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.457740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa54f8-61e5-4746-a578-485ca0bea4f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:23.459282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aa54f8-61e5-4746-a578-485ca0bea4f6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:23.461116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 53/104] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:23.462049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbe0c6-1a2f-4f29-bd96-7bc4f85cd408 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.462340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbe0c6-1a2f-4f29-bd96-7bc4f85cd408 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.465490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdbe0c6-1a2f-4f29-bd96-7bc4f85cd408 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.469991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 54/105] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:23.470821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbe0c6-1a2f-4f29-bd96-7bc4f85cd408 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:23.471783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f6d09-409a-4837-b68b-57bfd9f7eb91 None admin] RBAC: Authorizing `identity:get_user(user_id=164e98ad65ed4afb9e8c7c6d9fdb53db)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:23.476506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f6d09-409a-4837-b68b-57bfd9f7eb91 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:23.478373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 52/106] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:23 2022] GET /identity/v3/users/164e98ad65ed4afb9e8c7c6d9fdb53db => generated 289 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:23.480312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.480602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.480884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.500390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:23.521191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:23.522473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:23.598172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdbe0c6-1a2f-4f29-bd96-7bc4f85cd408 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:23.669894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:23.671547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.672618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.673319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.675502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:23.677634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:23.682546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 53/107] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:23.683807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:23.689351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:23.750698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 55/108] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:51:23 2022] POST /identity/v3/projects => generated 302 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:23.818922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac643ee5-5136-4c2f-8c9d-1564fc52e09b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:23.858107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1f7b5-4524-41c2-8350-8dc7d3bfa090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:23.858733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f7b5-4524-41c2-8350-8dc7d3bfa090 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:23.859021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f7b5-4524-41c2-8350-8dc7d3bfa090 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:23.859296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1f7b5-4524-41c2-8350-8dc7d3bfa090 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:23.862157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:23.938453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1f7b5-4524-41c2-8350-8dc7d3bfa090 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:23.939970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1f7b5-4524-41c2-8350-8dc7d3bfa090 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:24.049374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 54/109] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:24.054788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:24.078786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61690-89ca-4f94-842a-51a77617cb1a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.079074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61690-89ca-4f94-842a-51a77617cb1a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.079363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb61690-89ca-4f94-842a-51a77617cb1a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.080880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 55/110] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:24.092031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.092031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.092031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.112055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:24.112055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:24.152032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:24.221776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 56/111] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:24.236689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3302e1-32ab-4b69-8669-f6813c40d90c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:24.247878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:24.286524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:24.287684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.288203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.289563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.291007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:24.292930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:24.295454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 56/112] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:24 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:24.304704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:24.427257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb0cf4-5268-4130-9f82-80765c7b1cb5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:24.445035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:24.499470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2404553-d9db-41cd-ac30-2f19347031d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:24.500132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2404553-d9db-41cd-ac30-2f19347031d3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.500419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2404553-d9db-41cd-ac30-2f19347031d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.500695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2404553-d9db-41cd-ac30-2f19347031d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.515514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2404553-d9db-41cd-ac30-2f19347031d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c2404553-d9db-41cd-ac30-2f19347031d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:24.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:24.522071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:24.522821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 57/113] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:24.524317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8062bf1-70a9-4748-b5b8-fafbc637127c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.528100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8062bf1-70a9-4748-b5b8-fafbc637127c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.528100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8062bf1-70a9-4748-b5b8-fafbc637127c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.528100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 58/114] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:24.531146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8062bf1-70a9-4748-b5b8-fafbc637127c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:24.546088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bae517-b52c-4896-9d92-d3595de973d9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:24.580457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:24.598676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:24.599587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.600134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.600550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.601454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] RBAC: Adding query filter params (role.id=1321d256574b445aafa336e44bcd6562, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=164e98ad65ed4afb9e8c7c6d9fdb53db) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:24.601906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:24.603502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6a540-28e4-443d-9598-4cee7617732a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:24.610406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 57/115] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:24 2022] GET /identity/v3/role_assignments?user.id=164e98ad65ed4afb9e8c7c6d9fdb53db&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=1321d256574b445aafa336e44bcd6562 => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:24.611789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.612338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.612784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.641984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:24.643444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:24.734240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 58/116] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:24.735361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:24.789391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8062bf1-70a9-4748-b5b8-fafbc637127c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:24.803501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:24.884885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:24.885447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.885733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.886014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.886704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:24.887019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:24.888465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:24.900738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ff797-12fe-42ef-85cf-e14b540cd007 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:24.907365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 59/117] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:24.913131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:24.978480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6a958-a9fc-4d2a-95fa-97bb878e8038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:24.979132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6a958-a9fc-4d2a-95fa-97bb878e8038 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:24.979423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6a958-a9fc-4d2a-95fa-97bb878e8038 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:24.979708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6a958-a9fc-4d2a-95fa-97bb878e8038 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:24.991852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d6a958-a9fc-4d2a-95fa-97bb878e8038 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-e0d6a958-a9fc-4d2a-95fa-97bb878e8038 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:24.996977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:24.997882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:24.998651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 59/118] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:25.000039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361841-37d2-44e2-911e-cb068341971f None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.000366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361841-37d2-44e2-911e-cb068341971f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.000658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361841-37d2-44e2-911e-cb068341971f None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.001944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 60/119] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:25.006934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1361841-37d2-44e2-911e-cb068341971f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.116545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:25.119425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1361841-37d2-44e2-911e-cb068341971f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.132352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:25.134691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2199cc5-e21a-43be-a8ab-1d8763e7ecf6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.166795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.167423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.168085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.168436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.169141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:25.169524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:25.170847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad38451-0f88-4124-88a1-70d47a6a422d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:25.172756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.173402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.173684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.173961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.175149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:25.176703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:25.181771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 61/120] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:25.183046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 60/121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:24 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:25.183685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.183924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.184546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.185110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.211257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:25.212533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:25.305399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75570b4-755f-4c76-8497-4217ef3a356d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.314480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 62/122] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:25.315622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.372947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.373974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.377528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.378037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.399253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:25.405537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:25.406385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:25.407136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:25.413427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 61/123] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:25.414477 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.448685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56106-17a5-4e48-b4ff-13b0f7dfe250 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.449169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:25.451358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:25.510453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.511476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.511960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.512458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.515503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:25.517584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:25.521602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 63/124] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:25.524959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.638607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937dee8a-7ac7-4b39-b556-4070c63b1036 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.674870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.676567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.677043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.677503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.677916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a9409-c8ab-43d6-8d0a-ee96b3f9dff8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.678563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:25.679072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:25.680669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:25.689306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 64/125] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:25.689968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.716258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.717333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.717815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.718287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.719977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:25.721712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:25.723078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:25.725743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 62/126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:25.726748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.805143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc73a2-27a4-4130-9bd1-2a9e011a998f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.837753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:25.841234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.841939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.842219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.842495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554/users/940a93c8f7514875b6777c4fd2aaa85a/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.844882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e17314c11454ebc92e55967eee83554, user_id=940a93c8f7514875b6777c4fd2aaa85a, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:25.846346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:25.863959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 65/127] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:25 2022] PUT /identity/v3/projects/8e17314c11454ebc92e55967eee83554/users/940a93c8f7514875b6777c4fd2aaa85a/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:25.864539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daddf99-80c2-4b10-a871-1c2ab6cc81eb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:25.864793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:25.916302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:25.917337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:25.917823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:25.918596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:25.933904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:25.941604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:25.942540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:25.943315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:25.946350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 63/128] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:25.947571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.004093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:26.020958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30fed-24cc-4f83-8a16-71b060cd881a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.099313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.100606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.100892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.101171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.102454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.103936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.105644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 66/129] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:26.106458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.125381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579c749-ea72-410b-8a27-8e4b0bfb2340 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.164899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.165580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.165861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.166139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.169232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.170725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.172516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 64/130] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:25 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:26.173423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.200908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:26.248644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49588a-5bda-438f-99d6-486a687468e0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.333059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.337752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.338544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.339046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.340480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:26.341049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.342623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.359842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 67/131] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:26.360993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.398912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:26.454484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0ecc2-df79-4314-8766-78b5199fbba3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.497180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ee50f5-b67e-4ee5-9b5f-848b1e82cd5f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.542066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.543090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.543595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.544098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554/users/a29c05a14dda47f2a0f073398ac0dadb/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.547238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] RBAC: Authorizing `identity:create_grant(project_id=8e17314c11454ebc92e55967eee83554, user_id=a29c05a14dda47f2a0f073398ac0dadb, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.550531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.561446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.562482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.562969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.563641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] PATH_INFO: `/v3/projects/546c1d0af93e410292a37a557b0b45f6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.567187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] RBAC: Authorizing `identity:get_project(project_id=546c1d0af93e410292a37a557b0b45f6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.569605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.575290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 65/132] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:26 2022] GET /identity/v3/projects/546c1d0af93e410292a37a557b0b45f6 => generated 312 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:26.576367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.576724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 68/133] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:26 2022] PUT /identity/v3/projects/8e17314c11454ebc92e55967eee83554/users/a29c05a14dda47f2a0f073398ac0dadb/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:26.578537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.728764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:26.754735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609ba0e-7841-412b-8764-fa4937b6cb27 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.760405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:26.798181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7a74d-677d-419e-8727-7cf5106858a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.798579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb95a-d20a-485c-88b3-bcc2b4441ebe None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:26.799071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7a74d-677d-419e-8727-7cf5106858a8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.799452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7a74d-677d-419e-8727-7cf5106858a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.799855 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7a74d-677d-419e-8727-7cf5106858a8 None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.806131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7a74d-677d-419e-8727-7cf5106858a8 None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.811191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7a74d-677d-419e-8727-7cf5106858a8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.818966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 66/134] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:26 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:26.820349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a635-f062-40cd-baf0-2f398ca5a59b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.820641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a635-f062-40cd-baf0-2f398ca5a59b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.820922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a635-f062-40cd-baf0-2f398ca5a59b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.822322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 67/135] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:26.826139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a635-f062-40cd-baf0-2f398ca5a59b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:26.871852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab2f86-15a3-4443-afb5-48d62b16c621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:26.872980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab2f86-15a3-4443-afb5-48d62b16c621 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.873463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab2f86-15a3-4443-afb5-48d62b16c621 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.873937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab2f86-15a3-4443-afb5-48d62b16c621 None admin] PATH_INFO: `/v3/projects/546c1d0af93e410292a37a557b0b45f6/users/940a93c8f7514875b6777c4fd2aaa85a/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.877344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab2f86-15a3-4443-afb5-48d62b16c621 None admin] RBAC: Authorizing `identity:create_grant(project_id=546c1d0af93e410292a37a557b0b45f6, user_id=940a93c8f7514875b6777c4fd2aaa85a, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:26.879607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab2f86-15a3-4443-afb5-48d62b16c621 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:26.896071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 69/136] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:26 2022] PUT /identity/v3/projects/546c1d0af93e410292a37a557b0b45f6/users/940a93c8f7514875b6777c4fd2aaa85a/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:26.897856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512975c2-f4f2-4033-bee4-3ed5534cee6f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:26.898390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512975c2-f4f2-4033-bee4-3ed5534cee6f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:26.898998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512975c2-f4f2-4033-bee4-3ed5534cee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:26.935830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-512975c2-f4f2-4033-bee4-3ed5534cee6f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:26.937470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-512975c2-f4f2-4033-bee4-3ed5534cee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:27.010384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a635-f062-40cd-baf0-2f398ca5a59b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:27.083363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:27.088574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.088862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.089143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.090491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:27.091038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 70/137] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:27.092333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:27.092776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312742-13fe-4711-8ea2-010a11a08b6d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.093516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312742-13fe-4711-8ea2-010a11a08b6d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.093983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf312742-13fe-4711-8ea2-010a11a08b6d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.094284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 68/138] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:26 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:27.095770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 71/139] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:27.101170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:27.109489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.110080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.110570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.143238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:27.146035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:27.147534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:27.264756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:27.270395 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 72/140] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:27.328208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c287217-6ca3-49e1-b378-4890f20cd40d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:27.369644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:27.370676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.371177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.371687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554/users/940a93c8f7514875b6777c4fd2aaa85a/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.374285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] RBAC: Authorizing `identity:create_grant(project_id=8e17314c11454ebc92e55967eee83554, user_id=940a93c8f7514875b6777c4fd2aaa85a, role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:27.376132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:27.393152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 69/141] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:27 2022] PUT /identity/v3/projects/8e17314c11454ebc92e55967eee83554/users/940a93c8f7514875b6777c4fd2aaa85a/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:27.476326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:27.621124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bba9ae-7a7c-4126-9f6f-a54b8db10354 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:27.655581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:27.685508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:27.686180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.686465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.686737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.689384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:27.691406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:27.693083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 73/142] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:27.699296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:27.799455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:27.823439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6f1fa-31a3-4d72-9389-a47180a2c45f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:27.855562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:27.868488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:27.869408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.869829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.870245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.871074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:27.872771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:27.899947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e876c878-5aba-46b2-820c-502fc9ce752a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:27.907926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 74/143] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:27 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:27.961484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:27.962474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:27.962915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:27.963334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:27.980850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:27.986707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:27.988533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:27.989492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:27.991813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 70/144] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:27.998224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:28.127063 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9573a6-9c8b-46cd-9d2e-4f615c6f1a9f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:28.155837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:28.169534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:28.170678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:28.170962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:28.171241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:28.171926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:28.172318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:28.173793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:28.183601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 75/145] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:28.189687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:28.332523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335fb15-39bd-4834-98be-755e2e4388d2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:28.379644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:28.380244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:28.380527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:28.380801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] PATH_INFO: `/v3/users/164e98ad65ed4afb9e8c7c6d9fdb53db` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:28.381959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] RBAC: Authorizing `identity:get_user(user_id=164e98ad65ed4afb9e8c7c6d9fdb53db)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:28.383430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:28.385127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 71/146] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:28 2022] GET /identity/v3/users/164e98ad65ed4afb9e8c7c6d9fdb53db => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:28.391191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:28.544045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82733c76-c90d-4fe9-9ead-3265db918f08 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:28.598354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177153e9-196d-46b3-bac5-1c4c5481772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:28.599086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177153e9-196d-46b3-bac5-1c4c5481772e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:28.599520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177153e9-196d-46b3-bac5-1c4c5481772e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:28.599805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177153e9-196d-46b3-bac5-1c4c5481772e None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:28.602067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177153e9-196d-46b3-bac5-1c4c5481772e None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:28.603611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177153e9-196d-46b3-bac5-1c4c5481772e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:28.605645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 76/147] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:28 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:28.611801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:28.618590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:28.753852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3b5a09-a129-4be6-8456-9ac909ac69a5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:28.797752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b4c1c-2b86-4f97-8289-7582e16377ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:28.798446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b4c1c-2b86-4f97-8289-7582e16377ac None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:28.798725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b4c1c-2b86-4f97-8289-7582e16377ac None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:28.799000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b4c1c-2b86-4f97-8289-7582e16377ac None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554/users/164e98ad65ed4afb9e8c7c6d9fdb53db/roles/1321d256574b445aafa336e44bcd6562` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:28.801219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b4c1c-2b86-4f97-8289-7582e16377ac None admin] RBAC: Authorizing `identity:create_grant(project_id=8e17314c11454ebc92e55967eee83554, user_id=164e98ad65ed4afb9e8c7c6d9fdb53db, role_id=1321d256574b445aafa336e44bcd6562)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:28.802729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b4c1c-2b86-4f97-8289-7582e16377ac None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:28.822530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 72/148] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:28 2022] PUT /identity/v3/projects/8e17314c11454ebc92e55967eee83554/users/164e98ad65ed4afb9e8c7c6d9fdb53db/roles/1321d256574b445aafa336e44bcd6562 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:29.201345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e082-714f-4e3a-8ef3-8970d316da76 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:29.201844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e082-714f-4e3a-8ef3-8970d316da76 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:29.201926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e082-714f-4e3a-8ef3-8970d316da76 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:29.203500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 77/149] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:29.219627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ad11-0a85-4c1e-a148-6010e3841c50 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:29.219920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ad11-0a85-4c1e-a148-6010e3841c50 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:29.220219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95ad11-0a85-4c1e-a148-6010e3841c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:29.244754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9c95ad11-0a85-4c1e-a148-6010e3841c50 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:29.245957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9c95ad11-0a85-4c1e-a148-6010e3841c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:29.323689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:29.340714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 73/150] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:29.838816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f9050-ce8a-4fe3-a3be-e446e05dcab2 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:29.839460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f9050-ce8a-4fe3-a3be-e446e05dcab2 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:29.839959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f9050-ce8a-4fe3-a3be-e446e05dcab2 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:29.841534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 78/151] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:29.848045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fc4c-f0d7-4e15-bb59-04f8f0f83b6b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:29.848933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fc4c-f0d7-4e15-bb59-04f8f0f83b6b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:29.849426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444fc4c-f0d7-4e15-bb59-04f8f0f83b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:29.885066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5444fc4c-f0d7-4e15-bb59-04f8f0f83b6b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:29.886731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5444fc4c-f0d7-4e15-bb59-04f8f0f83b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:29.921763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1158f5-ba05-4fd1-a348-c2432c342c15 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:29.922367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1158f5-ba05-4fd1-a348-c2432c342c15 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:29.922834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1158f5-ba05-4fd1-a348-c2432c342c15 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:29.924692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 79/152] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:29.928535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1158f5-ba05-4fd1-a348-c2432c342c15 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:30.029376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 74/153] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:30.031921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:30.032818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:30.033614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:30.040166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:30.072563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:30.074088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:30.121836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1158f5-ba05-4fd1-a348-c2432c342c15 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:30.179253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 75/154] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:30.180733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:30.181457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:30.181806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:30.182151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:30.199093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:30.207323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:30.208661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:30.209810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 80/155] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:30.217779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:30.290313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:30.349110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9380d2-f6fc-4e05-9fa2-6bef72daf4f5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:30.394609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:30.395231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:30.395514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:30.395782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:30.396492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:30.396802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:30.398208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:30.407294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 76/156] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:30.413840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:30.574069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21bb1c-afff-4f27-8aa3-2af630dec87b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:30.591345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:30.614685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:30.615688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:30.616213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:30.616696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:30.618105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:30.619882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:30.621712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 81/157] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:30 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:30.627500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:30.736905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:30.778516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081442fe-8f4e-413e-9f2c-43835f546764 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:30.809727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:30.838285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bdca5-573c-4471-b50a-e012d50efe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:30.838822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bdca5-573c-4471-b50a-e012d50efe59 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:30.839104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bdca5-573c-4471-b50a-e012d50efe59 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:30.839381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bdca5-573c-4471-b50a-e012d50efe59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:30.935284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b9817-4eac-40a3-98b2-936b93a7b39e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:30.975805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bdca5-573c-4471-b50a-e012d50efe59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:30.978406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bdca5-573c-4471-b50a-e012d50efe59 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:31.006915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.007826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.008222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.008781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.036652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:31.042055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:31.043219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 82/158] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:31.044189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.103728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 77/159] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:31.105961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8efa4-873a-48e6-8e47-3d46b7fb0fd1 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.106393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8efa4-873a-48e6-8e47-3d46b7fb0fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.106805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8efa4-873a-48e6-8e47-3d46b7fb0fd1 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.108355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 78/160] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:31.109223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8efa4-873a-48e6-8e47-3d46b7fb0fd1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.148809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:31.170712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:31.231290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21a940-3cb7-4145-ac0c-9dba8fd55689 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.297815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.298646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.299024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.299394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.304544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8efa4-873a-48e6-8e47-3d46b7fb0fd1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.312342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:31.318271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:31.319256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:31.320049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:31.322528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 83/161] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:31.323721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.379041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.379571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.379847 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.380139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.380811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:31.381114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:31.382497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e1a860-9326-46ae-93c2-3fb1570c9ac8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:31.398545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 79/162] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:31.399795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.400098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.400529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.424333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:31.425902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:31.465711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a5f6c-a586-4b8a-b61a-45c6d5a2a4ac None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.467287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:31.508944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.510020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.510475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.510946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.512417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:31.514103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:31.516169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 84/163] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:31.517454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.525024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 80/164] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:31.526003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.548451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:31.634005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41475a6-0e55-431f-90d4-a595e1fbfbab None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.698378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.699422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.699712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.699993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.700707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:31.701017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:31.702896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:31.720453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 85/165] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:31.721343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.741744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61f686-005b-44d9-a278-29d5e9504602 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.792070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:31.827013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.827677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.827942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.828239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.829500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:31.831104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:31.833006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 81/166] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:31.833884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:31.898773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f482520-36b9-46d6-afaa-dccaed1fb0d2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.957496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:31.970060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:31.970554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:31.970903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:31.971251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:31.972218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=940a93c8f7514875b6777c4fd2aaa85a) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:31.972600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:31.974146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:31.991330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173b1a6-5b23-4e67-8319-d0c59090a7c0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:31.994184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 86/167] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/role_assignments?user.id=940a93c8f7514875b6777c4fd2aaa85a&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:31.995609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.041075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.042658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.043489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.044411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] PATH_INFO: `/v3/users/a29c05a14dda47f2a0f073398ac0dadb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.047213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] RBAC: Authorizing `identity:get_user(user_id=a29c05a14dda47f2a0f073398ac0dadb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:32.050447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:32.053363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 82/168] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/users/a29c05a14dda47f2a0f073398ac0dadb => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:32.059127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.198877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b34e8-2f2c-4c4a-bc85-75e0cbe27b97 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:32.231795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:32.239773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dbaf14-f1d6-435f-806f-35f1eefe3477 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:32.241367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.242247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.242741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.243229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.258705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:32.277258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.277872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.278155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.278428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.322130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:32.324253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:32.358077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:32.359707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:32.467858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:32.472466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 87/169] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:32.473535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.510799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 83/170] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:51:32.511995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.576372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:32.603746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677b68a-015c-499b-95ef-f3cb0e7ce743 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:32.641964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.642792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.643182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.644091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.647655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb4e7a-73f9-4d3f-ad45-9afa9d0e09d1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:32.656425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:32.662475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:32.663438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 88/171] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:32.664413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.691624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b69081-94e1-49ea-b11d-b3a25878ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.692579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b69081-94e1-49ea-b11d-b3a25878ee23 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.693027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b69081-94e1-49ea-b11d-b3a25878ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.693487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b69081-94e1-49ea-b11d-b3a25878ee23 None admin] PATH_INFO: `/v3/projects/546c1d0af93e410292a37a557b0b45f6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.694841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b69081-94e1-49ea-b11d-b3a25878ee23 None admin] RBAC: Authorizing `identity:get_project(project_id=546c1d0af93e410292a37a557b0b45f6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:32.696594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b69081-94e1-49ea-b11d-b3a25878ee23 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:32.698456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 84/172] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:32 2022] GET /identity/v3/projects/546c1d0af93e410292a37a557b0b45f6 => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:32.699866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bc12-914f-4fcc-94c9-53f23ec219d8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.700356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bc12-914f-4fcc-94c9-53f23ec219d8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.700829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460bc12-914f-4fcc-94c9-53f23ec219d8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.702250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 85/173] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:32.703208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460bc12-914f-4fcc-94c9-53f23ec219d8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.808364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:32.840974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460bc12-914f-4fcc-94c9-53f23ec219d8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:32.855627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:32.891645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d57635-6d28-42c9-8d6d-10cc0f7ebc7f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:32.891833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.892333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.892615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.892892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.894343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:32.894654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:32.896865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:32.909712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 86/174] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:32.910652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:32.932984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98274552-eeeb-4ed8-8bc9-4e0f3d40663c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:32.933623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98274552-eeeb-4ed8-8bc9-4e0f3d40663c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.933903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98274552-eeeb-4ed8-8bc9-4e0f3d40663c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.934175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98274552-eeeb-4ed8-8bc9-4e0f3d40663c None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.935313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98274552-eeeb-4ed8-8bc9-4e0f3d40663c None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:32.936800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98274552-eeeb-4ed8-8bc9-4e0f3d40663c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:32.938622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 89/175] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:32 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:32.939609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8c82-402d-4cfd-a165-6155d156b6b5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:32.939899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8c82-402d-4cfd-a165-6155d156b6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:32.940753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d8c82-402d-4cfd-a165-6155d156b6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:32.960418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-201d8c82-402d-4cfd-a165-6155d156b6b5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:32.961751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-201d8c82-402d-4cfd-a165-6155d156b6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:33.029583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c5226-f336-4579-bc26-f80ce6a0fe16 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:33.035550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 90/176] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:33.044090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1993e5-ab1f-4f1b-a65b-199eb440206e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.044090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1993e5-ab1f-4f1b-a65b-199eb440206e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.044090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1993e5-ab1f-4f1b-a65b-199eb440206e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.044090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 91/177] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:33.044090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1993e5-ab1f-4f1b-a65b-199eb440206e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:33.076919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:33.077803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.078084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.078389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.079725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=546c1d0af93e410292a37a557b0b45f6, user.id=940a93c8f7514875b6777c4fd2aaa85a) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:33.080123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:33.082536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e39947-0d2b-4769-bc12-8b99795980f5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:33.089390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 87/178] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:32 2022] GET /identity/v3/role_assignments?user.id=940a93c8f7514875b6777c4fd2aaa85a&scope.project.id=546c1d0af93e410292a37a557b0b45f6&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:33.090796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a5ec-6267-4d0f-9c15-fbb64647f237 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.091634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a5ec-6267-4d0f-9c15-fbb64647f237 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.091918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41a5ec-6267-4d0f-9c15-fbb64647f237 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.093629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 88/179] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:33.098036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a5ec-6267-4d0f-9c15-fbb64647f237 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:33.123466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:33.156063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:33.238624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41a5ec-6267-4d0f-9c15-fbb64647f237 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:33.247011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1993e5-ab1f-4f1b-a65b-199eb440206e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:33.291535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c5b33-bd9e-459a-8320-2fa5dff30481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:33.292741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c5b33-bd9e-459a-8320-2fa5dff30481 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.292741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c5b33-bd9e-459a-8320-2fa5dff30481 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.293054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c5b33-bd9e-459a-8320-2fa5dff30481 None admin] PATH_INFO: `/v3/users/940a93c8f7514875b6777c4fd2aaa85a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.294179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c5b33-bd9e-459a-8320-2fa5dff30481 None admin] RBAC: Authorizing `identity:get_user(user_id=940a93c8f7514875b6777c4fd2aaa85a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:33.295581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c5b33-bd9e-459a-8320-2fa5dff30481 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:33.297751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 92/180] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/users/940a93c8f7514875b6777c4fd2aaa85a => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:33.304149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.304496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.304764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.318867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:33.320554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.322401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.323253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.325397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=a29c05a14dda47f2a0f073398ac0dadb) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:33.326311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:33.328229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:33.329445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:33.330391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c31c40-9fcb-4435-929e-9645787f6287 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:33.354395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 89/181] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/role_assignments?user.id=a29c05a14dda47f2a0f073398ac0dadb&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:33.355705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.355994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.356302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.382851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:33.384317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:33.429682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 93/182] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:33.430635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:33.481928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 90/183] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:33.489094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:33.504771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:33.541909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:33.557862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1398d97-6dfb-4bd1-8751-6bcabdc870ab None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:33.599052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a3758-5f23-44b1-879a-f7b74529c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:33.599771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a3758-5f23-44b1-879a-f7b74529c379 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.600083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a3758-5f23-44b1-879a-f7b74529c379 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.600364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a3758-5f23-44b1-879a-f7b74529c379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.622947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e30c43a-67a7-49c5-96e4-243faa0972d2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:33.664293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cddd7f-501a-4f65-bb37-559990b90dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:33.664935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cddd7f-501a-4f65-bb37-559990b90dce None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.665214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cddd7f-501a-4f65-bb37-559990b90dce None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.665494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cddd7f-501a-4f65-bb37-559990b90dce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.676619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cddd7f-501a-4f65-bb37-559990b90dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:33.680507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a3758-5f23-44b1-879a-f7b74529c379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b2cddd7f-501a-4f65-bb37-559990b90dce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:33.681727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:33.682607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:33.683394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:33.683999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a3758-5f23-44b1-879a-f7b74529c379 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:33.684965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 91/184] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:33.684965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394ef3-397e-4756-8946-ff270bcb2d33 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.684965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394ef3-397e-4756-8946-ff270bcb2d33 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.684965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61394ef3-397e-4756-8946-ff270bcb2d33 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.688035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 92/185] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:33.689996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394ef3-397e-4756-8946-ff270bcb2d33 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:33.764387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:33.826429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 94/186] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:33.827721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.828032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.828337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.832840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:33.846313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61394ef3-397e-4756-8946-ff270bcb2d33 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:33.850375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:33.851537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:33.883184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:33.883786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:33.884093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:33.884365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:33.885023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:33.885321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:33.886695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:33.894675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 93/187] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:33.899943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:33.918875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 95/188] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:33.919787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.006762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24380bc2-5ae8-4381-b2c2-cc8b714b428f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.013428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.033746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac455ee4-0e2a-4564-9e02-a1f85eefff0b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.043322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.043957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.044268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.044541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.054800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:34.060280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:34.061157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:34.061973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 94/189] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:34.062642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.074466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.075186 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.075537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.075880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.077133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.078570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.080270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 96/190] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:33 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.081539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.088034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.187175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ae485-8624-49fe-ade8-1979e43fda0f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.190843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8b9f-2e33-4d81-aa73-2b669c221d0a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.222954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.223581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.223856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.224163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.226535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.227157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.227437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.227732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] PATH_INFO: `/v3/users/9c2efc576f514e869b08cbacfb48d0e8` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.234073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:34.239461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:34.240454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 95/191] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:34.242108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.246314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] RBAC: Authorizing `identity:get_user(user_id=9c2efc576f514e869b08cbacfb48d0e8)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.247931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.249880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 97/192] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/users/9c2efc576f514e869b08cbacfb48d0e8 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.250601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.262806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.314135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.345543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb3c46d-eaa5-4fe2-bc81-c1514164c05f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.361353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b431dd-4c52-45ec-af01-4b6106e6850c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.382419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.383444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.383729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.384193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.384867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:34.385176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.386560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.395862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 96/193] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.396828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.399845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.400506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.400782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.401080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.401807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] RBAC: Adding query filter params (role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=940a93c8f7514875b6777c4fd2aaa85a) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:34.402110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.403537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.413464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 98/194] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/role_assignments?user.id=940a93c8f7514875b6777c4fd2aaa85a&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.414281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.498202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.528112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000661-7e34-4cb5-953e-381789a9cae5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.540606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6942a6-30c1-4947-9611-d36ef59097dd None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.569882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.570803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.571096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.571379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.572830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.574339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.577619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 97/195] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.581613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.581613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.590426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.591040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.591316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.591586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.593537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:34.593679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.595058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.607351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 99/196] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.607909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.696256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42820538-90e2-4247-b50f-351b6878a331 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.718818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0969-b53b-4178-b870-20eb075c0056 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.736407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.737849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.738210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.738539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.755723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.756384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.756669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.756946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] PATH_INFO: `/v3/users/164e98ad65ed4afb9e8c7c6d9fdb53db` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.758083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] RBAC: Authorizing `identity:get_user(user_id=164e98ad65ed4afb9e8c7c6d9fdb53db)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.759557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.761209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 100/197] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/users/164e98ad65ed4afb9e8c7c6d9fdb53db => generated 289 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:34.761972 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.764994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.820263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.821751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.823603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:34.872465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8b493-b06a-46b3-85ea-e9d81c5948d1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:34.908028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:34.908652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:34.909602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:34.910090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:34.910872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:34.912369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:34.928692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 98/198] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:34.929582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:34.935869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 101/199] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:34 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:34.936660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.006619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.051298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2758c-f8d9-4411-a9d5-7b925163491d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.053648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.062016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30c9c1-6f67-497d-a483-88b907f38af2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.100399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.100399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.100399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.100399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] PATH_INFO: `/v3/users/9c2efc576f514e869b08cbacfb48d0e8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.102968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2efc576f514e869b08cbacfb48d0e8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.102968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.103116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.103454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.103730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.104085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 99/200] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/users/9c2efc576f514e869b08cbacfb48d0e8 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:35.104913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.108189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.193335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.193867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.232236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa9d15-863f-463d-b15b-2ea8e6a77fe9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.269288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.282452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.283099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.283368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.283694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.285045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.286672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.288588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 100/201] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:35.289144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.354703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.356402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 102/202] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:35.357805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.429824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73faff0a-2d1d-4127-a355-807a4d3aec25 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.465944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26072fcb-841d-4543-9fdd-8eab4e7a5f23 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.469909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.470588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.470941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.471304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.502066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.503283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.503572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.503847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.504628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=9c2efc576f514e869b08cbacfb48d0e8) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:35.504938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.506447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.512454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 103/203] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/role_assignments?user.id=9c2efc576f514e869b08cbacfb48d0e8&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:35.513290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.540520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.550918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.552437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.600330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.629780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579f4af-20bd-499d-bbd6-8ad89db497cc None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.675742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 101/204] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:35.681066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.681734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.681867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.682040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.682282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.683600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.685264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.687414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 104/205] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:35.704978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.798930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42883f-9b42-4110-883e-d09e3da5e9ab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.850111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.850754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.851030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.851304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.852466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.853932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.855586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 102/206] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:35.864765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:35.867250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.880894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea400e5-f57e-4c3f-9702-2612cac21530 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:35.884995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:35.921372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:35.922424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:35.922808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:35.923208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:35.927221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] RBAC: Adding query filter params (role.id=1321d256574b445aafa336e44bcd6562, scope.project.id=8e17314c11454ebc92e55967eee83554, user.id=164e98ad65ed4afb9e8c7c6d9fdb53db) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:35.927221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:35.928131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f7981-8e3b-4e40-96e7-d05e1d11ee07 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:35.936600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 105/207] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/role_assignments?user.id=164e98ad65ed4afb9e8c7c6d9fdb53db&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=1321d256574b445aafa336e44bcd6562 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:36.057823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cae16-8e08-4271-902e-4c5801204dcf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:36.111154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:36.111785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:36.112789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:36.113074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:36.113823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] RBAC: Adding query filter params (role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=9c2efc576f514e869b08cbacfb48d0e8) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:36.114136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:36.115631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005d699-ec68-4871-a607-d4c9a87e9118 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:36.126098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 103/208] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:35 2022] GET /identity/v3/role_assignments?user.id=9c2efc576f514e869b08cbacfb48d0e8&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:37.213632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e4819-2d17-4f01-98b3-23832684e080 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.214152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e4819-2d17-4f01-98b3-23832684e080 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.214255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e4819-2d17-4f01-98b3-23832684e080 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.215532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 106/209] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:37.221042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbaa4c-ef59-49f0-8900-88a0a3b2a6f4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.221611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbaa4c-ef59-49f0-8900-88a0a3b2a6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.221611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbaa4c-ef59-49f0-8900-88a0a3b2a6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.240612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-48bbaa4c-ef59-49f0-8900-88a0a3b2a6f4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:37.241815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-48bbaa4c-ef59-49f0-8900-88a0a3b2a6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:37.308649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:37.320180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 104/210] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:37.336101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc977e85-9eb3-46f1-8cf0-8afc3a7ce055 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.336385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc977e85-9eb3-46f1-8cf0-8afc3a7ce055 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.336662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc977e85-9eb3-46f1-8cf0-8afc3a7ce055 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.338298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 107/211] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:37.343747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc7414-1218-4cf3-81b6-0abc37d76dcb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.344076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc7414-1218-4cf3-81b6-0abc37d76dcb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.344368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afc7414-1218-4cf3-81b6-0abc37d76dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.364234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4afc7414-1218-4cf3-81b6-0abc37d76dcb None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:37.365472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4afc7414-1218-4cf3-81b6-0abc37d76dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:37.433433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 105/212] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:37.594942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc977e85-9eb3-46f1-8cf0-8afc3a7ce055 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:37.690642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20722fd0-1954-4fab-9c88-98ef31ed2a05 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.691090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20722fd0-1954-4fab-9c88-98ef31ed2a05 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.691485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20722fd0-1954-4fab-9c88-98ef31ed2a05 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.693264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 106/213] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:37.698223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.698499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.698785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.704820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc977e85-9eb3-46f1-8cf0-8afc3a7ce055 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:37.719445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:37.721372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:37.748638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:37.749406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.749687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.749948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.751193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:37.753314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:37.755281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 108/214] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:37.761116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:37.792460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:37.808118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 107/215] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:37.809290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:37.892062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70416b9-131b-49cd-bdfe-c1695aa96a9b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:37.892877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:37.937857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c6507-9db5-4a0a-9006-84d4d6a39a66 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:37.938281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:37.939046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.939466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.940657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:37.941456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:37.943845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:37.975988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 109/216] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:37 2022] POST /identity/v3/users => generated 297 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:37.991030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:37.991993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:37.992568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:37.992987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.007756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:38.013307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:38.014462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:38.015471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:38.023001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 108/217] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:38.024749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.097645 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.121687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.150712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a60187-07b6-4014-aba7-314685a3a531 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.194961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.195584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.195859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.196161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.197218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:38.197218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:38.198471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5647a3-582f-4b62-adb3-518f323d5029 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:38.207678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 110/218] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:38.208755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655ca0-d066-4ac4-a62a-c7b958403475 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.208977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655ca0-d066-4ac4-a62a-c7b958403475 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.209247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a655ca0-d066-4ac4-a62a-c7b958403475 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.210575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 111/219] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:38.215120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655ca0-d066-4ac4-a62a-c7b958403475 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.216480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f72ba-70e0-41c3-a49b-8016972e92cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.241012 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.262530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f82700-e728-407c-9a7c-61de6d70f469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.263155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f82700-e728-407c-9a7c-61de6d70f469 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.263434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f82700-e728-407c-9a7c-61de6d70f469 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.263710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f82700-e728-407c-9a7c-61de6d70f469 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.285315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f82700-e728-407c-9a7c-61de6d70f469 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a6f82700-e728-407c-9a7c-61de6d70f469 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:38.290310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:38.291224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:38.292034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 109/220] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:38.293021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.293301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.293699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.314238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:38.316378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:38.341075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a655ca0-d066-4ac4-a62a-c7b958403475 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.385950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.386737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.387121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.387497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] PATH_INFO: `/v3/users/c59cb2918e5d4215aaa9ecba76d39a9d` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.407792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 110/221] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:38.408522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.412322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.417717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] RBAC: Authorizing `identity:get_user(user_id=c59cb2918e5d4215aaa9ecba76d39a9d)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:38.419539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:38.421772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 112/222] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/users/c59cb2918e5d4215aaa9ecba76d39a9d => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:38.428577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.577549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfb543-6881-448d-957c-b06e531ee790 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.589047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.596772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777213b9-7b1c-47c6-a812-091e018d7b76 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.628618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.629706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.630187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.630665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.631534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:38.632056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:38.633660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:38.643580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 111/223] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:38.644823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.645668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.646162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.646626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.649239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.740326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.744610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:38.746269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:38.772626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d70b1b-f316-455c-a4dc-96f538f303a6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.811905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.812745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.813124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.813495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] PATH_INFO: `/v3/users/9c2efc576f514e869b08cbacfb48d0e8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.814759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2efc576f514e869b08cbacfb48d0e8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:38.816919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:38.819014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 112/224] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/users/9c2efc576f514e869b08cbacfb48d0e8 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:38.820062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.854676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 113/225] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:38.855751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:38.870930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.935146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8c967-f812-4efb-aa5a-0f67588ecdb1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.959793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05745ea-33c5-430f-8134-55b7f7a49cac None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:38.970397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:38.973123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:38.973874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:38.974255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:38.974631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:38.986265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:38.991989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:38.992949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:38.993731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 113/226] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:38.994375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.007600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.008548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.009023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.009489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.010860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.012509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.014396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 114/227] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:39.015369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.105267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:39.107171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a50027-4b1b-4ccb-a507-1566cc4658e2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.132662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605dde39-08e5-451c-b618-e14d3d047956 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.145308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.145949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.146312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.146481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.147629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.149188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.151006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 114/228] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:38 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:39.156415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.172339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.173384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.173852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.174903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.175831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:39.176349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.177975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.187087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 115/229] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:39.192718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.224949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:39.267170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b26068-8319-4496-9aa0-f9e4bf258cb1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.304637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.305307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.305526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ce8834-a3c8-4363-83b1-43da959c432c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.305628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.305893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/9c2efc576f514e869b08cbacfb48d0e8/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.307819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff6481d416344218c56ae7f832d9bbc, user_id=9c2efc576f514e869b08cbacfb48d0e8, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.309460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.324776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 115/230] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:39 2022] PUT /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/9c2efc576f514e869b08cbacfb48d0e8/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:39.325858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.345334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.346969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.347463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.347933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.348951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=c59cb2918e5d4215aaa9ecba76d39a9d) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:39.349472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.351264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.358236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 116/231] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:39 2022] GET /identity/v3/role_assignments?user.id=c59cb2918e5d4215aaa9ecba76d39a9d&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:39.368688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:39.494826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a7d9a-73a5-4696-a1a0-dde7536824c2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.536043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.537062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.537544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.538014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] PATH_INFO: `/v3/users/9c2efc576f514e869b08cbacfb48d0e8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.539500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2efc576f514e869b08cbacfb48d0e8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.541269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.543612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 116/232] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:39 2022] GET /identity/v3/users/9c2efc576f514e869b08cbacfb48d0e8 => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:39.548987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.667356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:39.677993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1df7f-1287-4983-8a29-73518e1e88a2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.720762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9cb8f-3274-4390-9df1-71a086db06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.721513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9cb8f-3274-4390-9df1-71a086db06b6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.722069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9cb8f-3274-4390-9df1-71a086db06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.722408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9cb8f-3274-4390-9df1-71a086db06b6 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.723728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9cb8f-3274-4390-9df1-71a086db06b6 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.728974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d9cb8f-3274-4390-9df1-71a086db06b6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.731010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 117/233] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:39 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:39.736573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:39.884304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:39.892056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bcf41-8fb8-43c9-862c-5bf450ded1f7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:39.931328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b817664-f3d0-4391-8db8-c8209e3948dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:39.931998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b817664-f3d0-4391-8db8-c8209e3948dc None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:39.932303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b817664-f3d0-4391-8db8-c8209e3948dc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:39.932576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b817664-f3d0-4391-8db8-c8209e3948dc None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/9c2efc576f514e869b08cbacfb48d0e8/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:39.934817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b817664-f3d0-4391-8db8-c8209e3948dc None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff6481d416344218c56ae7f832d9bbc, user_id=9c2efc576f514e869b08cbacfb48d0e8, role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:39.936309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b817664-f3d0-4391-8db8-c8209e3948dc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:39.954530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 117/234] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:39 2022] PUT /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/9c2efc576f514e869b08cbacfb48d0e8/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:40.353433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dceb4-1ca6-4359-a910-ed68ea89a7ce None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:40.353939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dceb4-1ca6-4359-a910-ed68ea89a7ce None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:40.354042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839dceb4-1ca6-4359-a910-ed68ea89a7ce None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:40.355502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 118/235] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:40.360952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:40.362462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:40.362944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:40.383888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:40.387067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed7881-cdcf-4f52-8782-bedcd0af6c48 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:40.387386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:40.387792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed7881-cdcf-4f52-8782-bedcd0af6c48 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:40.388294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed7881-cdcf-4f52-8782-bedcd0af6c48 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:40.393004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 119/236] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:40.405849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:40.406425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:40.406894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:40.428482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:40.430146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:40.490769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 118/237] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:40.522407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 120/238] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:40.899341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:40.934008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:41.069251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:41.071622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:41.083045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d562e-50a4-43de-8670-c2c7e8049f6c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:41.103751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc594bab-483e-4c9b-8794-ce9cae4a0bd3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:41.120902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:41.122270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.122768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.123236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.124584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:41.126734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:41.129582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 121/239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:41.134736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:41.236793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:41.237518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.237719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.237996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.249961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500acb9-8570-4800-998f-74cb672439a0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:41.290803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:41.291803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.292429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.292931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.293787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:41.295475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:41.296678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:41.301806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:41.302706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:41.303494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 119/240] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:41.317045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:41.338981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 122/241] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:41 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:41.448180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30c8ad-2157-4231-99f9-e9b7fc42740a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:41.490928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:41.491916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.492424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.492891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.493838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:41.494363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:41.495936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:41.505886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 120/242] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:41.511967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:41.531478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:41.656437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb62f0-292f-47bc-bde3-6d2e58f4f377 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:41.711665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d47f8-ffbf-4504-8319-616c10fb3543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:41.712357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d47f8-ffbf-4504-8319-616c10fb3543 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.712644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d47f8-ffbf-4504-8319-616c10fb3543 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.712977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d47f8-ffbf-4504-8319-616c10fb3543 None admin] PATH_INFO: `/v3/users/4fee08e2d93f4922ad829ae905d0e67b` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.731175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d47f8-ffbf-4504-8319-616c10fb3543 None admin] RBAC: Authorizing `identity:get_user(user_id=4fee08e2d93f4922ad829ae905d0e67b)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:41.732696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d47f8-ffbf-4504-8319-616c10fb3543 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:41.734643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 123/243] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:41 2022] GET /identity/v3/users/4fee08e2d93f4922ad829ae905d0e67b => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:41.740463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:41.752909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:41.755421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06164f9-a8e2-4493-97e7-10d0939d230a None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.755704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06164f9-a8e2-4493-97e7-10d0939d230a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.755981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06164f9-a8e2-4493-97e7-10d0939d230a None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.757921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 124/244] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:41.763330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbcdaf-1824-4867-b4bf-f17442f575a5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.763608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbcdaf-1824-4867-b4bf-f17442f575a5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.763885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbcdaf-1824-4867-b4bf-f17442f575a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.794397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c0dbcdaf-1824-4867-b4bf-f17442f575a5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:41.795855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c0dbcdaf-1824-4867-b4bf-f17442f575a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:41.853676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d81293-36d8-471a-b897-c776a7721cba None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:41.883318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 125/245] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:41.885084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee214de-55a2-4912-90f0-9d26c9d9b1d2 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.885629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee214de-55a2-4912-90f0-9d26c9d9b1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.886122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee214de-55a2-4912-90f0-9d26c9d9b1d2 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.888238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 126/246] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:41.890344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:41.890932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.891208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.891475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.894833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:41.895674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:41.896401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:41.920796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:41.922544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:41.928510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:41.969978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:41.972212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:41.982925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:42.008825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 127/247] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:42.089468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 121/248] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:42.095378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.224243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c65d4-f0f9-4ac6-9098-27b2d9d66604 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:42.233893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.263081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:42.263541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.263828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.264131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.265641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:42.267229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:42.270275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 128/249] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:42.270842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:42.278199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.353827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a5e4-05da-4365-b989-9ef8a1251700 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:42.364821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:42.403333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:42.404403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.404830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.405218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.414904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44418d4-2e4b-4766-8a14-0aabad782739 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:42.421753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:42.430375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:42.431346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:42.432167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 122/250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:42.433643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.464296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:42.464959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.465246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.465538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.466400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] RBAC: Adding query filter params (role.id=1321d256574b445aafa336e44bcd6562, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=4fee08e2d93f4922ad829ae905d0e67b) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:42.466712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:42.468609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:42.478038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 129/251] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/role_assignments?user.id=4fee08e2d93f4922ad829ae905d0e67b&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=1321d256574b445aafa336e44bcd6562 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:42.481280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.606162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:42.615612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94680aa-9501-4c5e-88fc-3fd590e58ee1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:42.677929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb44621-1924-4714-bb08-13c29769036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:42.678927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb44621-1924-4714-bb08-13c29769036d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.679353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb44621-1924-4714-bb08-13c29769036d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.679797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb44621-1924-4714-bb08-13c29769036d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.691444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d6edf-cc5b-4d6f-9aac-35d740eda492 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:42.692509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb44621-1924-4714-bb08-13c29769036d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ebb44621-1924-4714-bb08-13c29769036d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:42.698672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:42.699606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:42.700428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:42.708123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 123/252] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:42.709862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2aa9-16d3-43e2-8f5b-e4e698db1ce0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.710413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2aa9-16d3-43e2-8f5b-e4e698db1ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.710894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2aa9-16d3-43e2-8f5b-e4e698db1ce0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.712570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 124/253] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:42.714755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2aa9-16d3-43e2-8f5b-e4e698db1ce0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.732046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:42.732645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.732925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.733201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.734021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:42.734199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:42.735542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b2e68-6e98-4e2c-861d-ae7ae124fb3f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:42.746597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 130/254] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:42.747661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.747942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.750895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:42.753030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.774402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:42.775661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:42.867921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2aa9-16d3-43e2-8f5b-e4e698db1ce0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:42.904592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 131/255] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:42.905500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:42.929715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:42.930436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:42.930715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:42.930981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:42.931649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:42.932028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:42.933456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:42.945513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 125/256] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:42.950972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:43.050097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:43.099229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:43.109088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69584b74-1844-4d54-b957-9e7e52e37579 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:43.183325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:43.184121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:43.184404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:43.184678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] PATH_INFO: `/v3/users/9c2efc576f514e869b08cbacfb48d0e8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:43.185509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740de208-431f-4f2a-92f2-6b5923aa8ebd None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:43.185911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2efc576f514e869b08cbacfb48d0e8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:43.187553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:43.191081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 126/257] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/users/9c2efc576f514e869b08cbacfb48d0e8 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:43.200151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:43.290311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:43.291358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:43.291840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:43.292356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] PATH_INFO: `/v3/users/c59cb2918e5d4215aaa9ecba76d39a9d` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:43.293938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] RBAC: Authorizing `identity:get_user(user_id=c59cb2918e5d4215aaa9ecba76d39a9d)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:43.295624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:43.298387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 132/258] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:42 2022] GET /identity/v3/users/c59cb2918e5d4215aaa9ecba76d39a9d => generated 297 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:43.300358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:43.390384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3d1f4-75b7-45d0-811a-c2d7a4a9f9d0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:43.454662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e539f95-ca62-4e3c-87f0-9f6ea1ebd9b5 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:43.479424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:43.480077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:43.480362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:43.480646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:43.482163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:43.508151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:43.508811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:43.509087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:43.509354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:43.529509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:43.538197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:43.539142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:43.540443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 133/259] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:43.541120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:43.544295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:43.589176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:43.590884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:43.757131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 127/260] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:43.761415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:43.785926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16347853-b70e-45af-8b6d-6fc4de1749e8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:43.836380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:43.859983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:43.860704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:43.860993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:43.861273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:43.863807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:43.865415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:43.867284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 134/261] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:43 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:43.872986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:43.908996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad58e01-5198-4791-b7c4-19d03df0eaf4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:43.968778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:43.969414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:43.969700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:43.969982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:43.970666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:43.971534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:43.971726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:43.973322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:43.991630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 128/262] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:43.992350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:44.030158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d9253f-eed2-4a21-8086-30092e2e2b14 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:44.068704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:44.069831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.070365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.070828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.072181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:44.073869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:44.076038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 135/263] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:43 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:44.077097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:44.194576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae6e15e-767b-4aa5-8051-164577dacdee None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:44.218504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:44.226212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4128b3ab-2e90-4d69-a9b2-aee9e46d85bb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:44.235024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:44.236057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.236577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.237044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] PATH_INFO: `/v3/users/9c2efc576f514e869b08cbacfb48d0e8` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.238559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] RBAC: Authorizing `identity:get_user(user_id=9c2efc576f514e869b08cbacfb48d0e8)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:44.240270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:44.242523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 136/264] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:44 2022] GET /identity/v3/users/9c2efc576f514e869b08cbacfb48d0e8 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:44.243496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:44.278734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:44.313974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:44.314418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.315232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.315509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/c59cb2918e5d4215aaa9ecba76d39a9d/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.321927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff6481d416344218c56ae7f832d9bbc, user_id=c59cb2918e5d4215aaa9ecba76d39a9d, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:44.323407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:44.344915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 129/265] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:43 2022] PUT /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/c59cb2918e5d4215aaa9ecba76d39a9d/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:44.345817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:44.389266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2de3fe-87e5-45e2-967a-2f172f7c0a63 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:44.443350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:44.443984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.444351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.444693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.445385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=9c2efc576f514e869b08cbacfb48d0e8) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:44.452376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:44.454037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e3ab74-4e20-41e8-bea6-908a4331e533 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:44.466482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 137/266] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:44 2022] GET /identity/v3/role_assignments?user.id=9c2efc576f514e869b08cbacfb48d0e8&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:44.468128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a5d7d-ad7a-4389-a4ab-0824b4c13dab None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.468469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a5d7d-ad7a-4389-a4ab-0824b4c13dab None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.468747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a5d7d-ad7a-4389-a4ab-0824b4c13dab None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.470170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 138/267] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:44.481901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.482212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.482450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.530629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:44.531836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:44.551177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64025eea-91b0-4df4-b980-566af8418609 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:44.615496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:44.619573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba2fea-ef6a-47ee-a2f5-314be723035f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:44.620327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba2fea-ef6a-47ee-a2f5-314be723035f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:44.620653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba2fea-ef6a-47ee-a2f5-314be723035f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:44.620923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba2fea-ef6a-47ee-a2f5-314be723035f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:44.661595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 139/268] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:44.718192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:44.752184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba2fea-ef6a-47ee-a2f5-314be723035f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:44.753678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba2fea-ef6a-47ee-a2f5-314be723035f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:44.889042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 130/269] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:44.894644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.001909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a4145-330a-4322-b8e1-a8d978ee0b99 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.002741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a4145-330a-4322-b8e1-a8d978ee0b99 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.003439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8a4145-330a-4322-b8e1-a8d978ee0b99 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.005476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 131/270] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:45.010457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb320ef-cdf3-4df7-b5a6-b41920f0d592 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.010742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb320ef-cdf3-4df7-b5a6-b41920f0d592 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.011016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb320ef-cdf3-4df7-b5a6-b41920f0d592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.033017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d95559-9bc6-4222-b0e6-888132b3fdc8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.051772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ebb320ef-cdf3-4df7-b5a6-b41920f0d592 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:45.053034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ebb320ef-cdf3-4df7-b5a6-b41920f0d592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:45.110862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2095df-6273-45ba-b55d-7ea318e1c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.111734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2095df-6273-45ba-b55d-7ea318e1c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.112131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2095df-6273-45ba-b55d-7ea318e1c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.112617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2095df-6273-45ba-b55d-7ea318e1c9a0 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.113873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2095df-6273-45ba-b55d-7ea318e1c9a0 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:45.115385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2095df-6273-45ba-b55d-7ea318e1c9a0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:45.117510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 140/271] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:44 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:45.118618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15667b8-5c83-463c-a08d-8cc1dbaa0e4e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.118903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15667b8-5c83-463c-a08d-8cc1dbaa0e4e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.119178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15667b8-5c83-463c-a08d-8cc1dbaa0e4e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.120592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 141/272] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:45.121157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15667b8-5c83-463c-a08d-8cc1dbaa0e4e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.161618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 132/273] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:45.162857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.163082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.163356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.182564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:45.183884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:45.185376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:45.188112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:45.259016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15667b8-5c83-463c-a08d-8cc1dbaa0e4e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.268991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 133/274] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:45.277181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.306096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.306743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.307029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.307304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.323321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:45.329187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:45.330886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:45.331820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 142/275] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:45.337557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.462547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada18e44-6fbe-4df6-8b09-49f2618d356a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.477928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48813d89-7228-49fe-837b-79bd421bb926 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.493456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:45.506850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.507445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.507819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.508214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.509124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] RBAC: Adding query filter params (role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=9c2efc576f514e869b08cbacfb48d0e8) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:45.509464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:45.511022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:45.524112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 134/276] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/role_assignments?user.id=9c2efc576f514e869b08cbacfb48d0e8&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:45.525838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.531680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.532619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.535245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.535720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.536640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:45.537097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:45.537667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:45.546206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:45.570560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 143/277] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:45.585910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.725975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23330c-b0ed-4552-ab9b-e8258d987a2f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.755878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91aa4f-4ea3-4e08-b769-c7b98efd17a0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.777190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.778426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.778901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.779531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.796835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:45.801946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:45.802940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:45.803728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:45.804361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.804361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.804361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.804568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 135/278] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:45.804634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] PATH_INFO: `/v3/users/a7b058fc343f4ea1bcfa57963d0c4f2e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.804699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.825219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] RBAC: Authorizing `identity:get_user(user_id=a7b058fc343f4ea1bcfa57963d0c4f2e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:45.826835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:45.829039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 144/279] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/users/a7b058fc343f4ea1bcfa57963d0c4f2e => generated 293 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:45.833074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.869255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:45.887113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:45.931772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0362821f-aa28-4714-948b-b43c74c0dcd4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.948654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303d1aad-811d-44b4-806b-5a1b0f9237cc None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:45.973730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.974485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.974785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.975147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.976337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:45.977931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:45.979775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 136/280] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:45.985109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:45.986077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:45.987247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:45.987534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:45.987804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:45.988511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:45.988817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:45.990201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:45.999341 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 145/281] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:46.000277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:46.098332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:46.106281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e27463b-01b1-4b97-a11c-857d50f89904 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:46.108474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381ba9b-9fe1-43d4-97ed-03582b97098f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:46.143863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:46.149419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:46.150479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.150761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.151067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.151763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, user.id=a7b058fc343f4ea1bcfa57963d0c4f2e, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:46.152105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:46.152353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:46.152844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.153127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.153401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.153559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:46.154055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:46.155735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:46.159303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 137/282] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 10:51:45 2022] GET /identity/v3/role_assignments?user.id=a7b058fc343f4ea1bcfa57963d0c4f2e&scope.system=all&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:46.159974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:46.195674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 146/283] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:51:45 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:46.292583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fb95c-ed0b-40e6-b9ff-9fe2a16f3735 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:46.337007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:46.337665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.337939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.338206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] PATH_INFO: `/v3/users/4fee08e2d93f4922ad829ae905d0e67b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.339557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] RBAC: Authorizing `identity:get_user(user_id=4fee08e2d93f4922ad829ae905d0e67b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:46.341095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:46.343423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 138/284] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:46 2022] GET /identity/v3/users/4fee08e2d93f4922ad829ae905d0e67b => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:46.350050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:46.464092 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cd260-69ad-49b5-b02c-937acdbb134d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:46.516810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878cc9f-c8a2-43c4-93ce-8e4898d70166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:46.517482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878cc9f-c8a2-43c4-93ce-8e4898d70166 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.517825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878cc9f-c8a2-43c4-93ce-8e4898d70166 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.518105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878cc9f-c8a2-43c4-93ce-8e4898d70166 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.519369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878cc9f-c8a2-43c4-93ce-8e4898d70166 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:46.521406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878cc9f-c8a2-43c4-93ce-8e4898d70166 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:46.523166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 147/285] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:46 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:46.530871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:46.623246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:46.676392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46380999-fb72-4416-ac5d-66ef8c8c23bb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:46.679661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8a50f-8f81-4bb1-8f29-9999b216668d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.679952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8a50f-8f81-4bb1-8f29-9999b216668d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.680269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f8a50f-8f81-4bb1-8f29-9999b216668d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.682666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 148/286] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:46.688126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.688410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.688685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.718690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:46.720256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:46.748326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:46.757554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:46.758238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:46.758745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:46.759022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/4fee08e2d93f4922ad829ae905d0e67b/roles/1321d256574b445aafa336e44bcd6562` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:46.763254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff6481d416344218c56ae7f832d9bbc, user_id=4fee08e2d93f4922ad829ae905d0e67b, role_id=1321d256574b445aafa336e44bcd6562)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:46.764752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:46.790760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 139/287] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:51:46 2022] PUT /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc/users/4fee08e2d93f4922ad829ae905d0e67b/roles/1321d256574b445aafa336e44bcd6562 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:46.870920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 149/288] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:47.193284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:47.332210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609f3b1-f72c-4f58-9b0b-58e8e20393c6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:47.372498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:47.391529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:47.392213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:47.392573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:47.393356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:47.411610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:47.416603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:47.417467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:47.418340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 140/289] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:47.424331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:47.537381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb0e0c0-ae66-4cae-898d-931df9955bc5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:47.579560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:47.580630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:47.581109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:47.581858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:47.582723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:47.583215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:47.584870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:47.599229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 150/290] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:47.604999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:47.756466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c16115-050f-4bea-a32e-8bcf44d66430 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:47.808571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:47.809201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:47.809483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:47.809760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] PATH_INFO: `/v3/users/c59cb2918e5d4215aaa9ecba76d39a9d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:47.810994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] RBAC: Authorizing `identity:get_user(user_id=c59cb2918e5d4215aaa9ecba76d39a9d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:47.812492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:47.814417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 141/291] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:47 2022] GET /identity/v3/users/c59cb2918e5d4215aaa9ecba76d39a9d => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:47.820223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:47.836777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:47.976027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f143cf-319a-4b2f-8d5b-85c941af1466 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:48.029942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05b09f-a176-4b61-b78f-17e80bef5d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:48.030570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b09f-a176-4b61-b78f-17e80bef5d78 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:48.030857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b09f-a176-4b61-b78f-17e80bef5d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:48.031130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05b09f-a176-4b61-b78f-17e80bef5d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:48.128261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05b09f-a176-4b61-b78f-17e80bef5d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:48.129607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05b09f-a176-4b61-b78f-17e80bef5d78 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:48.145335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:48.265050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 151/292] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:48.270429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:48.313291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74ca5-09f4-40d4-a2e8-c47bb0077cfa None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:48.313801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74ca5-09f4-40d4-a2e8-c47bb0077cfa None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:48.314309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74ca5-09f4-40d4-a2e8-c47bb0077cfa None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:48.316844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 152/293] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:51:48.322133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:48.323010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:48.323773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:48.349096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:48.350563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:48.369986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:48.432120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 153/294] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:48.482002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b130ae5-ceb9-4320-863e-31c6451e401d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:48.532709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:48.533384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:48.533662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:48.533935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:48.540991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:48.542492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:48.544318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 142/295] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:48 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:48.551214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:48.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:48.681256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbafd9e-0a64-440e-889e-1c2e4384bbb8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:48.721484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:48.722278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:48.722569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:48.722845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:48.723607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=c59cb2918e5d4215aaa9ecba76d39a9d) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:48.723921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:48.725417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced714b4-fd6a-4f03-976e-3893ea0f9a9a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:48.732470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 154/296] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:48 2022] GET /identity/v3/role_assignments?user.id=c59cb2918e5d4215aaa9ecba76d39a9d&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:48.835233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:48.980949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880b02a2-62cd-47c4-951d-12561749e29c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.008884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:49.030573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa10bb7-1a1d-4c3c-be25-ae805172bdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.031586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10bb7-1a1d-4c3c-be25-ae805172bdc4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.032078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10bb7-1a1d-4c3c-be25-ae805172bdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.032540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa10bb7-1a1d-4c3c-be25-ae805172bdc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.047244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa10bb7-1a1d-4c3c-be25-ae805172bdc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:49.053582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9f510-ce82-41e7-a110-a5b3197a614e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.053582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9f510-ce82-41e7-a110-a5b3197a614e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.053582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c9f510-ce82-41e7-a110-a5b3197a614e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-daa10bb7-1a1d-4c3c-be25-ae805172bdc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:49.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:49.054699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:49.055488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:49.056647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 143/297] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:49.057031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 155/298] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:49.059491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153dfb7-4ac8-431b-a00e-fc0356ff4bd5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.059491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153dfb7-4ac8-431b-a00e-fc0356ff4bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.059491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153dfb7-4ac8-431b-a00e-fc0356ff4bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.063122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9f510-ce82-41e7-a110-a5b3197a614e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.085087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1153dfb7-4ac8-431b-a00e-fc0356ff4bd5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:49.086328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1153dfb7-4ac8-431b-a00e-fc0356ff4bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:49.166552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 144/299] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:49.172139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440c68-33a1-4d38-8761-396c3166b7ca None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.172139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440c68-33a1-4d38-8761-396c3166b7ca None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.172139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb440c68-33a1-4d38-8761-396c3166b7ca None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.172139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 145/300] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:49.176508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.176791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.177063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.199444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c9f510-ce82-41e7-a110-a5b3197a614e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.203565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:49.206119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:49.218221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:49.252345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.252898 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.253376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.253827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.254921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:49.255502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:49.257109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:49.267175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 156/301] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:49.281067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.300523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 146/302] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:49.397715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a07abf2-2764-4cb4-ae58-012294d0a216 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.434472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.435098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.435382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.435654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] PATH_INFO: `/v3/users/a7b058fc343f4ea1bcfa57963d0c4f2e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.436900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] RBAC: Authorizing `identity:get_user(user_id=a7b058fc343f4ea1bcfa57963d0c4f2e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:49.438365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:49.440949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 157/303] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:49 2022] GET /identity/v3/users/a7b058fc343f4ea1bcfa57963d0c4f2e => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:49.446642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.483232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.494712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:49.520689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:49.562806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0807e2-ae48-4571-b3df-5d78d3a3e29d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.600020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa55ef03-3f9f-4382-9581-f00fabf59e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.600993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55ef03-3f9f-4382-9581-f00fabf59e26 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.601106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c18c5-ce36-4ef3-a654-5df4e988e62f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.601229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55ef03-3f9f-4382-9581-f00fabf59e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.601229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa55ef03-3f9f-4382-9581-f00fabf59e26 None admin] PATH_INFO: `/v3/system/users/a7b058fc343f4ea1bcfa57963d0c4f2e/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.602671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55ef03-3f9f-4382-9581-f00fabf59e26 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a7b058fc343f4ea1bcfa57963d0c4f2e, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:49.604121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa55ef03-3f9f-4382-9581-f00fabf59e26 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:49.613537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 147/304] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Aug 31 10:51:49 2022] PUT /identity/v3/system/users/a7b058fc343f4ea1bcfa57963d0c4f2e/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:51:49.614856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dca30-fc6a-4701-a779-ebc46997bc2c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.615142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dca30-fc6a-4701-a779-ebc46997bc2c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.615417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3dca30-fc6a-4701-a779-ebc46997bc2c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.616891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 148/305] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:49.617710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dca30-fc6a-4701-a779-ebc46997bc2c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.649017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325ffbf-1cd4-4f54-b6b8-e00862b5153b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.649674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325ffbf-1cd4-4f54-b6b8-e00862b5153b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.650074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325ffbf-1cd4-4f54-b6b8-e00862b5153b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.650345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325ffbf-1cd4-4f54-b6b8-e00862b5153b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.651727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325ffbf-1cd4-4f54-b6b8-e00862b5153b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:49.653317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325ffbf-1cd4-4f54-b6b8-e00862b5153b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:49.660142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 158/306] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:49.661730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.662016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.662324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.683177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:49.684500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:49.740663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3dca30-fc6a-4701-a779-ebc46997bc2c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.743308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:49.770900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 159/307] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:49.771145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.788996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.789726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.790001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.790357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.805418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:49.810834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:49.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:49.812605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:49.813198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 149/308] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:49.813851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:49.819854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:49.910337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a068f7f-93f7-4622-949e-9d6b451cf480 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.938000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b12593-ca66-433b-b261-80a819a991b7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:49.956864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.957729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.958390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.958728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] PATH_INFO: `/v3/groups` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.959557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:49.961231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:49.979539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:49.980197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:49.980499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:49.980738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:49.981421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:49.981751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:49.983351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:49.985122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 160/309] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Aug 31 10:51:49 2022] POST /identity/v3/groups => generated 229 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:51:49.996702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 150/310] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:50.005362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:50.061731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:50.096055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:50.135681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:50.153526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d72d52-b72c-4ace-baf7-f680ad9e752d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:50.199668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf6e567-30ee-4e68-9a57-3aa49fe5d7c2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:50.204810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:50.205537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:50.205819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:50.206088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] PATH_INFO: `/v3/users/900c83fb064c4d489e471f2b29a55c5a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:50.225275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] RBAC: Authorizing `identity:get_user(user_id=900c83fb064c4d489e471f2b29a55c5a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:50.227496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:50.229733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 161/311] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/users/900c83fb064c4d489e471f2b29a55c5a => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:50.235634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:50.240057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:50.240703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:50.241085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:50.241461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:50.253247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:50.258998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:50.259902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:50.260728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 151/312] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:50.268660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:50.365179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:50.371024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2075b-8a21-49b5-bfa1-8a5f4bbd1c86 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:50.385186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0005b2d-6e7e-4209-9fce-2c911329badc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:50.411045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:50.411785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:50.412103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:50.412379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:50.413296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] RBAC: Adding query filter params (role.id=1321d256574b445aafa336e44bcd6562, user.id=900c83fb064c4d489e471f2b29a55c5a, scope.system=all) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:50.413523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:50.415020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:50.427339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 162/313] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/role_assignments?user.id=900c83fb064c4d489e471f2b29a55c5a&scope.system=all&role.id=1321d256574b445aafa336e44bcd6562 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:50.430817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:50.431762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:50.432284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:50.432811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:50.433751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:50.434268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:50.435948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:50.446647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 152/314] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:50.452843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:50.473624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:50.586803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81823e08-72a5-4a99-9178-11705aaa0d23 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:50.627653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:50.628667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:50.629148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:50.629638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] PATH_INFO: `/v3/users/4fee08e2d93f4922ad829ae905d0e67b` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:50.631032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] RBAC: Authorizing `identity:get_user(user_id=4fee08e2d93f4922ad829ae905d0e67b)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:50.632693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:50.634493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 163/315] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/users/4fee08e2d93f4922ad829ae905d0e67b => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:50.640383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:50.750003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb9a95-8d98-4e2b-bdfc-ce38dea94836 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:50.792035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:50.792035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:50.792035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:50.792035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:50.824419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:50.869154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:50.870624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:50.978254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 153/316] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:50.983645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:51.073016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:51.094858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06c4aa-41ab-4ee0-9d9e-4ec222f9e225 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:51.131506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5a486-ce46-4379-89d5-ada1cb769c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:51.132180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5a486-ce46-4379-89d5-ada1cb769c0a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:51.132461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5a486-ce46-4379-89d5-ada1cb769c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:51.132784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5a486-ce46-4379-89d5-ada1cb769c0a None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:51.133931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5a486-ce46-4379-89d5-ada1cb769c0a None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:51.135406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5a486-ce46-4379-89d5-ada1cb769c0a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:51.137120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 164/317] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:50 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:51.143965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:51.201401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:51.253080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9a121-0b5c-4222-b67e-397ee7594795 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:51.288188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:51.288805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:51.289089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:51.289364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:51.290098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] RBAC: Adding query filter params (role.id=1321d256574b445aafa336e44bcd6562, scope.project.id=9ff6481d416344218c56ae7f832d9bbc, user.id=4fee08e2d93f4922ad829ae905d0e67b) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:51.290410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:51.291864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715690a8-d30a-4acd-867a-f01b1c81bd02 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:51.298348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 154/318] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:51:51 2022] GET /identity/v3/role_assignments?user.id=4fee08e2d93f4922ad829ae905d0e67b&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=1321d256574b445aafa336e44bcd6562 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:51.662798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d02afb-0259-4cc7-9fce-f64eac5fbbae None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:51.663604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d02afb-0259-4cc7-9fce-f64eac5fbbae None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:51.663604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d02afb-0259-4cc7-9fce-f64eac5fbbae None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:51.664751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 165/319] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:51.669987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dfe0c-bf07-4280-bbce-6051e67eb968 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:51.670397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dfe0c-bf07-4280-bbce-6051e67eb968 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:51.670582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dfe0c-bf07-4280-bbce-6051e67eb968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:51.689377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9c4dfe0c-bf07-4280-bbce-6051e67eb968 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:51.690667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9c4dfe0c-bf07-4280-bbce-6051e67eb968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:51.761665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 155/320] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:52.034350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d02afb-0259-4cc7-9fce-f64eac5fbbae None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.037921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab90850-a484-4b28-927e-281e50fb84b0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.038212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab90850-a484-4b28-927e-281e50fb84b0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.038491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab90850-a484-4b28-927e-281e50fb84b0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.039865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 156/321] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:52.045121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.045408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.045687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.064473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:52.065665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:52.108272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:52.144454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 157/322] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:52.156742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d02afb-0259-4cc7-9fce-f64eac5fbbae None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.193373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.193997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.194272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.194538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.205710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:52.210482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:52.211359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:52.216920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 166/323] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:52.219151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.319796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba2ea8-709d-44da-8b59-2e57bdfd747e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.355681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.356321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.356598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.356868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.357527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:52.357828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:52.359206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02fb8a-a3a4-47a9-97bd-5ad86a93dc25 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:52.367944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 158/324] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:52.373284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.385699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:52.418125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e0c50-8722-4888-8aae-5cba2001052a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.418439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e0c50-8722-4888-8aae-5cba2001052a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.418685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e0c50-8722-4888-8aae-5cba2001052a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.420209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 159/325] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:52.425139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.425425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.425706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.443986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:52.445226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:52.474825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:52.479488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6279bd-b6b5-4b0b-be26-91e380d307b8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.514016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba6785-da44-4871-a805-709cb910514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.514648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba6785-da44-4871-a805-709cb910514c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.514927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba6785-da44-4871-a805-709cb910514c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.515263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba6785-da44-4871-a805-709cb910514c None admin] PATH_INFO: `/v3/users/a7b058fc343f4ea1bcfa57963d0c4f2e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.517360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba6785-da44-4871-a805-709cb910514c None admin] RBAC: Authorizing `identity:get_user(user_id=a7b058fc343f4ea1bcfa57963d0c4f2e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:52.517518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 160/326] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:52.517518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.518444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba6785-da44-4871-a805-709cb910514c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:52.520243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 167/327] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/users/a7b058fc343f4ea1bcfa57963d0c4f2e => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:52.525847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba6785-da44-4871-a805-709cb910514c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.621178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:52.625514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e746933-32de-4049-a11e-1960cfe97dbc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.650474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba6785-da44-4871-a805-709cb910514c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.663917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.664614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.664896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.665171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.666295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:52.668592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:52.670369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 161/328] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:52.675869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.693179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.693696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.693947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.694157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.694875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] RBAC: Adding query filter params (role.id=8249b61571f3472089a8d3cc13b0c9df, user.id=a7b058fc343f4ea1bcfa57963d0c4f2e, scope.system=all) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:52.695181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:52.696685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:52.710157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 168/329] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/role_assignments?user.id=a7b058fc343f4ea1bcfa57963d0c4f2e&scope.system=all&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:51:52.711878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:52.787219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:52.804419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f2dbb2-4688-48cb-9e57-d3657d0ef819 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.843439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.844101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.844380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.844655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] PATH_INFO: `/v3/groups` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.846370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:52.847850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:52.868053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 162/330] 10.222.0.20 () {60 vars in 1134 bytes} [Wed Aug 31 10:51:52 2022] POST /identity/v3/groups => generated 226 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:51:52.923239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d21e51-c3d3-47ab-bed9-3c8ca6b80343 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:52.963564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:52.964237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:52.964510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:52.964777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:52.966849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:52.977633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:52.983142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:52.984222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:52.984996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:52.986459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 169/331] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:51:52.991249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:53.086030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:53.096618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae17e52-4194-4b99-8847-8664c7cf36ac None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:53.137310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:53.137927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:53.138205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:53.138488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:53.139145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:53.139448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:53.140884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:53.150522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 163/332] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:53.156154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:53.263246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8603cf1-0423-4625-a0b0-c12dc0d14992 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:53.302837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb38430-519b-48eb-90d8-2d5f61903a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:53.303742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb38430-519b-48eb-90d8-2d5f61903a10 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:53.304201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb38430-519b-48eb-90d8-2d5f61903a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:53.304629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb38430-519b-48eb-90d8-2d5f61903a10 None admin] PATH_INFO: `/v3/users/900c83fb064c4d489e471f2b29a55c5a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:53.305809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb38430-519b-48eb-90d8-2d5f61903a10 None admin] RBAC: Authorizing `identity:get_user(user_id=900c83fb064c4d489e471f2b29a55c5a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:53.307271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb38430-519b-48eb-90d8-2d5f61903a10 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:53.309305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 170/333] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:53 2022] GET /identity/v3/users/900c83fb064c4d489e471f2b29a55c5a => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:53.315041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:53.427341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ca2ba7-9d18-4554-ba14-f6aa90e1365a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:53.467580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc89a5-1cc1-4d9b-adcc-58dcba5f150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:53.468353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc89a5-1cc1-4d9b-adcc-58dcba5f150f None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:53.468633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc89a5-1cc1-4d9b-adcc-58dcba5f150f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:53.468910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc89a5-1cc1-4d9b-adcc-58dcba5f150f None admin] PATH_INFO: `/v3/system/users/900c83fb064c4d489e471f2b29a55c5a/roles/1321d256574b445aafa336e44bcd6562` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:53.470956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc89a5-1cc1-4d9b-adcc-58dcba5f150f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=900c83fb064c4d489e471f2b29a55c5a, role_id=1321d256574b445aafa336e44bcd6562)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:53.472388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc89a5-1cc1-4d9b-adcc-58dcba5f150f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:53.488178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 164/334] 10.222.0.20 () {58 vars in 1334 bytes} [Wed Aug 31 10:51:53 2022] PUT /identity/v3/system/users/900c83fb064c4d489e471f2b29a55c5a/roles/1321d256574b445aafa336e44bcd6562 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:51:55.319828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c0b47-1ffe-4859-8598-f3e0d7550984 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:55.320856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c0b47-1ffe-4859-8598-f3e0d7550984 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:55.321337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c0b47-1ffe-4859-8598-f3e0d7550984 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:55.322919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 171/335] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:55.328699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f447422-e4b3-4ef9-9dfb-6bd1b5f29a5a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:55.330133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f447422-e4b3-4ef9-9dfb-6bd1b5f29a5a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:55.330614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f447422-e4b3-4ef9-9dfb-6bd1b5f29a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:55.357734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4f447422-e4b3-4ef9-9dfb-6bd1b5f29a5a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:55.359284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f447422-e4b3-4ef9-9dfb-6bd1b5f29a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:55.381338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:55.449106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 165/336] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:55.616316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c751d-0708-426b-a83f-c8254cc626cd None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:55.616611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c751d-0708-426b-a83f-c8254cc626cd None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:55.616793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c751d-0708-426b-a83f-c8254cc626cd None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:55.618472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 172/337] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:55.623814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:55.624408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:55.624897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:55.647196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:55.648771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:55.735172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893188c-25a8-4099-bff7-1c313e529f2c None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:55.735489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893188c-25a8-4099-bff7-1c313e529f2c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:55.736287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b893188c-25a8-4099-bff7-1c313e529f2c None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:55.737997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 173/338] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:55.743476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:55.743755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:55.744129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:55.744694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 166/339] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:55.779610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:55.780484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:55.865450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:55.882820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 174/340] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:51:55.969610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.069393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:56.092726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2424e9ba-71b8-4433-b599-cb32006cd478 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:56.144478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:56.145542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.146024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.146505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.163497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:56.168831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:56.169702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:56.170517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:56.173272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 167/341] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:56.180461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.285276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.325618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb07f2-ed41-4211-b350-dd7e6ffd5fc1 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:56.368497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:56.369120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.369404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.369677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.370370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:56.372212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:56.374098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:56.384864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 175/342] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:56.393599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.465748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:56.493405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4283876d-cd60-411f-9be8-7f97d33b65d2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:56.526962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14692044-20c0-4ac7-9ffd-8e91f15d904a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:56.534712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46825901-8e08-4050-b5e9-dade90c1d0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:56.535388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46825901-8e08-4050-b5e9-dade90c1d0e1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.535698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46825901-8e08-4050-b5e9-dade90c1d0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.536066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46825901-8e08-4050-b5e9-dade90c1d0e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.548088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:56.553882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46825901-8e08-4050-b5e9-dade90c1d0e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-46825901-8e08-4050-b5e9-dade90c1d0e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:56.560203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:56.561288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:56.562128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 168/343] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:56.565296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e07b1-01dd-4ec0-9d51-61a7cbd0343b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.565296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e07b1-01dd-4ec0-9d51-61a7cbd0343b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.565296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e07b1-01dd-4ec0-9d51-61a7cbd0343b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.565296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 169/344] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:56.565849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e07b1-01dd-4ec0-9d51-61a7cbd0343b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.577279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f9bbc-e446-4b54-9a8a-f7ebc59323c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:56.577945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f9bbc-e446-4b54-9a8a-f7ebc59323c4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.578226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f9bbc-e446-4b54-9a8a-f7ebc59323c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.578498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f9bbc-e446-4b54-9a8a-f7ebc59323c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.686719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f9bbc-e446-4b54-9a8a-f7ebc59323c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:56.688205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882f9bbc-e446-4b54-9a8a-f7ebc59323c4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:56.703837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e07b1-01dd-4ec0-9d51-61a7cbd0343b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:56.752985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:56.753601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.753881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.754154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.765912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:56.768066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:56.779171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:56.780199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:56.781088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:56.784164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 170/345] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:56.785090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.836278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 176/346] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:56.837622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.838082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.838548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.863248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:56.864573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:56.869515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:56.925025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb52e-e644-4daa-a82a-7a4e4ae0247e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:56.946412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 177/347] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:56.947307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:56.982177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:56.982944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:56.983560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:56.983838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:56.984840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:56.985148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:56.987153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:56.997986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 171/348] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:57.004869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.062651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.086217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a559-d04f-486e-b1c8-b9168b28ed7e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.134213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.135346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.135888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.136427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.137329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:57.137835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:57.138323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b7fc3-5c9b-42df-9b6a-31e21574ff6a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.139703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db2555-d356-4eaf-877d-50bb231543eb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:57.150939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 178/349] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:57.152570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08db9-40aa-40aa-a57f-7162e9ce4cb9 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.153083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08db9-40aa-40aa-a57f-7162e9ce4cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.153575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc08db9-40aa-40aa-a57f-7162e9ce4cb9 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.155468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 179/350] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:57.156680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc08db9-40aa-40aa-a57f-7162e9ce4cb9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.169397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.187851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3962a9-bff7-4c72-9e92-abc587d663ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.189484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3962a9-bff7-4c72-9e92-abc587d663ee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.189838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3962a9-bff7-4c72-9e92-abc587d663ee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.190119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3962a9-bff7-4c72-9e92-abc587d663ee None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.198520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3962a9-bff7-4c72-9e92-abc587d663ee None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:57.198520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3962a9-bff7-4c72-9e92-abc587d663ee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:57.198520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 172/351] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:56 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:57.198520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.198520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.198520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.223537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:57.227256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:57.308384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc08db9-40aa-40aa-a57f-7162e9ce4cb9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.320995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 173/352] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:57.321908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.351384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.351978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.352320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.352600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.371111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.447371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:57.451159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:57.460986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c692ee-f157-4a1d-b286-58277edac535 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.486755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.515739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.516406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.516666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.517502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.609886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 180/353] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:57.611133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.618917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:57.620640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:57.667923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.733729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9a007-b063-4362-b311-d2331b1bceb2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.774063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 174/354] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:57.775203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.783416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac94b5-c28a-4f10-9b6b-5cb33bddc733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.784145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac94b5-c28a-4f10-9b6b-5cb33bddc733 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.784423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac94b5-c28a-4f10-9b6b-5cb33bddc733 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.784689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac94b5-c28a-4f10-9b6b-5cb33bddc733 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.791653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac94b5-c28a-4f10-9b6b-5cb33bddc733 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:57.793147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac94b5-c28a-4f10-9b6b-5cb33bddc733 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:57.794822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 181/355] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:57.795989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b46a7-1ad3-4294-a48d-d86f2d944332 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.796289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b46a7-1ad3-4294-a48d-d86f2d944332 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.796560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b46a7-1ad3-4294-a48d-d86f2d944332 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.797228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.800131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 182/356] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:51:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:51:57.800131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b46a7-1ad3-4294-a48d-d86f2d944332 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.911903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19858697-7c11-4488-988e-73f717a0d838 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.928342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b46a7-1ad3-4294-a48d-d86f2d944332 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:57.942307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:57.965808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.966558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.967106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.967377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.976103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:57.976860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:57.977214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:57.977489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:57.979626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:57.979905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:57.981139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:57.982825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 183/357] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:57.983614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:57.984974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:57.987069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 175/358] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:57.987944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.087147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:58.116558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3434-5ec4-496a-adb4-a8456ed6feaf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.127445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9a760-1a25-4546-8b12-5cdebd44a4bf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.159016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.159652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.159931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.160271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.161566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.163200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.165040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 176/359] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.165654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.169108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41727b74-c143-4829-b9f4-bf5bd85d25c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.169982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41727b74-c143-4829-b9f4-bf5bd85d25c2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.170264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41727b74-c143-4829-b9f4-bf5bd85d25c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.170540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41727b74-c143-4829-b9f4-bf5bd85d25c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.184343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41727b74-c143-4829-b9f4-bf5bd85d25c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-41727b74-c143-4829-b9f4-bf5bd85d25c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:58.191279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:58.192333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:58.193154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 184/360] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:51:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:58.193849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.194006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.196081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.214949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:51:58.216197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:51:58.257694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:58.294631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c62a20-b87b-4931-bf8c-54a3ff373548 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.310919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 185/361] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:51:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:51:58.312455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.343168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.343955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.344444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.344723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.347282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=8e17314c11454ebc92e55967eee83554) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:58.347282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.348570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.354942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 177/362] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.355932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.373060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:58.478044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55ec0-4a96-4e24-a331-104d63c9f26c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.507233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40ddc9-dd89-429b-8a2a-a3cf872f9bfe None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.543310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.543986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.544314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.544587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.545716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.547151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.549055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 186/363] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.549754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.558875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.559622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.559979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.560298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.560983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:58.561295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.562690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.574360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 178/364] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.575098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.597216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:58.670316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:58.685825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33120dc9-e209-4b77-9b28-e6a062ee2340 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.727099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f516b0-93f1-4b94-9e0b-e70ad950932c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.746439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.747020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.747302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.747575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.748804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.750243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.752850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 187/365] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.753699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.776782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.777546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.777831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.778103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.778887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:58.779195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.783691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.799616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 179/366] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.802230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.902187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c284abe-a760-4476-a058-1e574ae4e7c1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.905650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:58.942520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09721976-3310-41ea-8b95-7bcdc22b6347 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:58.952539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.953157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.953433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.953700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.955086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=9ff6481d416344218c56ae7f832d9bbc) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:58.955599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:58.957284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:58.963356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 188/367] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:58.966053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:58.992805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:58.994146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:58.994648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:58.995126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] PATH_INFO: `/v3/users/900c83fb064c4d489e471f2b29a55c5a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:58.997698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] RBAC: Authorizing `identity:get_user(user_id=900c83fb064c4d489e471f2b29a55c5a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.000262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.003163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 180/368] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/users/900c83fb064c4d489e471f2b29a55c5a => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:59.004914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.008597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:59.105145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494dff67-0e49-4137-8ba4-3703e091bbdf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.152191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.153026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.153385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.154596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.164468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c6e04-a58e-4441-a1e4-fae51b240a85 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.174951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:51:59.180478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:51:59.181604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:51:59.182433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 189/369] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:51:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:51:59.183076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.217145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.217909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.218577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.218577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.220614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=8e17314c11454ebc92e55967eee83554) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:59.221470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.222936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.228508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:59.231318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 181/370] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:59.232208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.326188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:59.326990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0c0f2-6f39-4a6d-9a0b-13758dfc525d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.371445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a5ce5-5071-4d1b-8fe2-59df28413116 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.387693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.388602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.389026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.389441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.441719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.442432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.442832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.443530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.444036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] RBAC: Adding query filter params (role.id=1321d256574b445aafa336e44bcd6562, user.id=900c83fb064c4d489e471f2b29a55c5a, scope.system=all) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:59.444440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.446165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.470687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.472551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.473303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 182/371] 10.222.0.20 () {56 vars in 1308 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/role_assignments?user.id=900c83fb064c4d489e471f2b29a55c5a&scope.system=all&role.id=1321d256574b445aafa336e44bcd6562 => generated 510 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:59.474256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.520436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:59.606881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 190/372] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:51:59.613338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.645947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1368f2ff-ace3-4f2e-95a3-d14d8a513961 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.672902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:59.685972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.686594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.686877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.687154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.687837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:51:59.688178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.690523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.700400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 183/373] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:59.706469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.734590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b279c-6b4b-48a6-a184-45d027c76f2f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.768405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:51:59.779573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.780378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.783222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.783222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.783222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.785812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.788049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 191/374] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:51:59.793720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:51:59.887651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6b6a1d-9ec7-4ddc-8ba4-afbf3747f6c2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.924384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6280258-2a83-456a-bcd6-2334415aa0ae None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:51:59.955161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.956112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.956680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.957127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.992131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:51:59.992131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:51:59.992131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:51:59.993874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:51:59.995475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:51:59.997072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:51:59.998610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 192/375] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:00.006515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:00.045353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:00.076840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:00.080897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:00.141584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:00.238414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8cef8-8388-4f3d-af97-2cc604188886 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:00.289597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 184/376] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:52:00.295218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:00.298980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:00.299614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:00.299911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:00.300216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:00.300952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=9ff6481d416344218c56ae7f832d9bbc) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:00.301261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:00.302727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:00.314078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 193/377] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:00 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 260 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:00.410244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:00.428752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61acc8a-0e10-43ef-afb4-1e312f4176f6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:00.471739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:00.472434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:00.472716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:00.472991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] PATH_INFO: `/v3/projects/34b2b6cdb81943ad9b9237874eeca93b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:00.474205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] RBAC: Authorizing `identity:get_project(project_id=34b2b6cdb81943ad9b9237874eeca93b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:00.475679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:00.477724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 185/378] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:00 2022] GET /identity/v3/projects/34b2b6cdb81943ad9b9237874eeca93b => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:00.483551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:00.633632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93042c10-07bc-4581-a062-d0a0be8824a9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:00.665227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:00.677565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c6530-7d7a-4cee-baf3-0dcfd6673f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:00.678954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c6530-7d7a-4cee-baf3-0dcfd6673f35 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:00.679490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c6530-7d7a-4cee-baf3-0dcfd6673f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:00.680250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c6530-7d7a-4cee-baf3-0dcfd6673f35 None admin] PATH_INFO: `/v3/groups/fa42ce4197764dde835c642966f818a9` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:00.692145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c6530-7d7a-4cee-baf3-0dcfd6673f35 None admin] RBAC: Authorizing `identity:get_group(group_id=fa42ce4197764dde835c642966f818a9)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:00.695178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c6530-7d7a-4cee-baf3-0dcfd6673f35 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:00.698968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 194/379] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:00 2022] GET /identity/v3/groups/fa42ce4197764dde835c642966f818a9 => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:00.705452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:00.888552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa6811-bc40-45cb-8c08-506eb6064170 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:00.953917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:00.955939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:00.956720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:00.957496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:00.959162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] RBAC: Adding query filter params (group.id=fa42ce4197764dde835c642966f818a9, role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=34b2b6cdb81943ad9b9237874eeca93b) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:00.959997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:00.963046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc91ee-fd85-4500-b763-5b7ec806b4ed None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:00.974730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 186/380] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:00 2022] GET /identity/v3/role_assignments?group.id=fa42ce4197764dde835c642966f818a9&scope.project.id=34b2b6cdb81943ad9b9237874eeca93b&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:01.055591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428f2f6-2aa5-43cf-9c73-8655d577bb58 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.056634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428f2f6-2aa5-43cf-9c73-8655d577bb58 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.057398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4428f2f6-2aa5-43cf-9c73-8655d577bb58 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.060227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 195/381] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:01.065636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.066334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.066805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.093286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:01.095790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:01.111918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:01.187367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 187/382] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:01.500798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428f2f6-2aa5-43cf-9c73-8655d577bb58 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:01.633953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4428f2f6-2aa5-43cf-9c73-8655d577bb58 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:01.680472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d391dbc9-2876-422b-b986-84b29db0894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:01.681488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391dbc9-2876-422b-b986-84b29db0894d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.681993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391dbc9-2876-422b-b986-84b29db0894d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.682478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d391dbc9-2876-422b-b986-84b29db0894d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.695754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d391dbc9-2876-422b-b986-84b29db0894d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-d391dbc9-2876-422b-b986-84b29db0894d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:01.706940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:01.707939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:01.708783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:01.711628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 196/383] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:01.721250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:01.755659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c38c5-f1ae-475e-8ad8-94f757303dda None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.756321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c38c5-f1ae-475e-8ad8-94f757303dda None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.756798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c38c5-f1ae-475e-8ad8-94f757303dda None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.758746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 197/384] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:01.771870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd58c8-8048-463f-9789-33901bb72ff7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.772823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd58c8-8048-463f-9789-33901bb72ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.773852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd58c8-8048-463f-9789-33901bb72ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.801361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c4fd58c8-8048-463f-9789-33901bb72ff7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:01.802585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c4fd58c8-8048-463f-9789-33901bb72ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:01.884405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dfda63-d6b4-4a96-8a12-0f38c4cbc6c2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:01.892414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:01.921022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 198/385] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:52:01.922129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812f803-2bf1-41a1-bf98-2518f9342c34 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.922441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812f803-2bf1-41a1-bf98-2518f9342c34 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.922662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812f803-2bf1-41a1-bf98-2518f9342c34 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.927448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:01.928540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.928949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 199/386] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:01.929272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.929759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.930332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:01.930618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:01.930890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:01.931275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:01.931794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:01.933751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:01.944804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 188/387] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:52:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:01.956121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:01.956737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:01.957379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:01.983184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:02.039321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 200/388] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:02.106208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2eceb-3606-4c52-81a6-b425e442d1fb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:02.141142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:02.142376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:02.142855 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:02.143215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:02.144379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:02.145830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:02.147755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 189/389] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:01 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:02.153029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:02.305476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bff41e-ceb1-49a5-907f-f2b580f0d875 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:02.327238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:02.353184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:02.353829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:02.354111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:02.354391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:02.357262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:02.358770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:02.361440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 201/390] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:02 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:02.362809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:02.368405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:02.502824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e023f2c-e4da-48ed-81f9-347f62eedb0c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:02.512567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:02.546983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:02.548073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:02.548498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:02.548916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554/groups/a20ac3b450304e798825c539367360f0/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:02.550418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e52d1-852f-4aac-862b-90f8b5ae61e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:02.551166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] RBAC: Authorizing `identity:create_grant(project_id=8e17314c11454ebc92e55967eee83554, group_id=a20ac3b450304e798825c539367360f0, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:02.553041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:02.573614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 202/391] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 10:52:02 2022] PUT /identity/v3/projects/8e17314c11454ebc92e55967eee83554/groups/a20ac3b450304e798825c539367360f0/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:52:02.580407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:02.612389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:02.613041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:02.613325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:02.613603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:02.636736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:02.651263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:02.652569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:02.653647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 190/392] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:02.675954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:02.689636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:02.823818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25dbc7-dadb-4090-b29a-286f59878ccb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:02.862979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:02.864661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:02.865095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:02.865455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:02.877555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5cf58-8e75-43e2-974d-5959969b2fc5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:02.879045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:02.886812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:02.888028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:02.888874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:02.889507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 203/393] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:02.895915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:02.923747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:02.924609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:02.924991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:02.925339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:02.926177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:02.926557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:02.928960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:02.935617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:02.942248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 191/394] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:52:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:02.947916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.021457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.025137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b97e4e-2fcb-454f-8cde-928d257023b4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.067514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.068675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.068972 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.069249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.070350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:03.070661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.072325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02704829-d795-4e3e-9d14-0102b8f23903 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.075779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9281479-4fb1-4474-8616-fae912bb455d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.083360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 204/395] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:03.089184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda874-4e14-4034-94f5-f4c969ca89be None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.089476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda874-4e14-4034-94f5-f4c969ca89be None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.089753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eda874-4e14-4034-94f5-f4c969ca89be None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.091353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 205/396] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:03.092420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda874-4e14-4034-94f5-f4c969ca89be None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.115044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67eee6b-7b7c-411a-94b6-06b99ea9cfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.116066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eee6b-7b7c-411a-94b6-06b99ea9cfe9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.116606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eee6b-7b7c-411a-94b6-06b99ea9cfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.117082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67eee6b-7b7c-411a-94b6-06b99ea9cfe9 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.118520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eee6b-7b7c-411a-94b6-06b99ea9cfe9 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.120196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67eee6b-7b7c-411a-94b6-06b99ea9cfe9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.122323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 192/397] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:02 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:03.123694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.124201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.124702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.144463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:03.146205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:03.212218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.213270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eda874-4e14-4034-94f5-f4c969ca89be None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.220649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 193/398] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:03.221823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.253332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.253997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.254278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.254551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.256142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.257439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.259504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 206/399] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:03 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:03.264947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.302012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.328588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96fe4d-21a4-4a2e-acbd-484e8c4006f9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.365347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa047b9c-a82d-4eb9-8252-12c0dfe3c3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.366778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047b9c-a82d-4eb9-8252-12c0dfe3c3eb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.367257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047b9c-a82d-4eb9-8252-12c0dfe3c3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.367992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa047b9c-a82d-4eb9-8252-12c0dfe3c3eb None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.369499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa047b9c-a82d-4eb9-8252-12c0dfe3c3eb None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.371715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa047b9c-a82d-4eb9-8252-12c0dfe3c3eb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.374616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 194/400] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:03 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:03.376700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4bee-0a64-4a08-a124-042b7c79cf2d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.377200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4bee-0a64-4a08-a124-042b7c79cf2d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.377671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4bee-0a64-4a08-a124-042b7c79cf2d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.379368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 195/401] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:03.381426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a4bee-0a64-4a08-a124-042b7c79cf2d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.384083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413e06c-ee2c-4163-9798-d9b0f9f43f30 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.427074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803eb1d-cc20-48d0-b8b2-9230c85d5c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.427712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803eb1d-cc20-48d0-b8b2-9230c85d5c65 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.427990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803eb1d-cc20-48d0-b8b2-9230c85d5c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.428289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803eb1d-cc20-48d0-b8b2-9230c85d5c65 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.429425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803eb1d-cc20-48d0-b8b2-9230c85d5c65 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.430858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8803eb1d-cc20-48d0-b8b2-9230c85d5c65 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.432675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 207/402] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:03 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:03.433743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.434024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.434297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.453539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:03.454766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:03.465407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.503740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a4bee-0a64-4a08-a124-042b7c79cf2d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.530706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 208/403] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:03.531913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.539668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.540513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.540792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.541065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc/groups/a20ac3b450304e798825c539367360f0/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.543248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff6481d416344218c56ae7f832d9bbc, group_id=a20ac3b450304e798825c539367360f0, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.544971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.565014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.567356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 196/404] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 10:52:03 2022] PUT /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc/groups/a20ac3b450304e798825c539367360f0/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:52:03.573197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.717836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3e890-2264-4a4d-a39f-5875869a2f4f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.771490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f51392-8677-48f7-b7f5-a5cb163b6284 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:03.776493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.777170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.777450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.777725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554/groups/a20ac3b450304e798825c539367360f0/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.780652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] RBAC: Authorizing `identity:create_grant(project_id=8e17314c11454ebc92e55967eee83554, group_id=a20ac3b450304e798825c539367360f0, role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:03.782154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:03.803143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 209/405] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 10:52:03 2022] PUT /identity/v3/projects/8e17314c11454ebc92e55967eee83554/groups/a20ac3b450304e798825c539367360f0/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:03.814764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.825100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:03.825873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:03.826190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:03.826472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:03.837326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.843010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:03.851078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:03.852552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:03.853396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 197/406] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:03.859457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:03.981822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:03.996895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69052f70-6e5d-4699-8a51-9007bf22de37 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.011292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374a90-6192-4f3e-8433-5cb441630545 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.039959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.040642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.040960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.041187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.055140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:04.057455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.058065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.058348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.058621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.059287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:04.059589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:04.060360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:04.061228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:04.062019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:04.062621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.062695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 210/407] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:04.070308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.072632 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 198/408] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:04.078829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.147903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:04.235102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f797360-8794-4aa5-8b3a-135b15e5cb16 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.240118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c2837-4b12-4511-8678-7476714222bf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.313173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.313946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.314231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.314530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.316000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.317473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.318928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.319378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 199/409] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:04 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:04.320116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.320569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.321010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.321909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:04.322369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.324418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.325069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.343670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 211/410] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:04.350333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.372537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:04.522893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d0d60-457b-4f2b-bcdf-c97d7e874562 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.567882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.569103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.569379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.569653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] PATH_INFO: `/v3/groups/fa42ce4197764dde835c642966f818a9` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.573041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] RBAC: Authorizing `identity:get_group(group_id=fa42ce4197764dde835c642966f818a9)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.574385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.577286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 212/411] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:04 2022] GET /identity/v3/groups/fa42ce4197764dde835c642966f818a9 => generated 229 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:04.586711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.593017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06531-d58e-4702-ba2d-de92169b9cb5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.623148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:04.669324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.669981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.670264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.670537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.671770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.673521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.676955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 200/412] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:04 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:04.684681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.763014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:04.820905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dbcc44-b65d-4183-8914-c14bd4b27da4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.882499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15f30d-78a4-4011-8fee-7a3e4304bdc7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:04.924597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b80918-d248-40bb-902c-9eee69d7453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.924841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:04.925317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80918-d248-40bb-902c-9eee69d7453d None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.925438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:04.925727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80918-d248-40bb-902c-9eee69d7453d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.925855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:04.925933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b80918-d248-40bb-902c-9eee69d7453d None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc/groups/a20ac3b450304e798825c539367360f0/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.926012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] PATH_INFO: `/v3/projects/34b2b6cdb81943ad9b9237874eeca93b` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:04.927120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] RBAC: Authorizing `identity:get_project(project_id=34b2b6cdb81943ad9b9237874eeca93b)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.928206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b80918-d248-40bb-902c-9eee69d7453d None admin] RBAC: Authorizing `identity:create_grant(project_id=9ff6481d416344218c56ae7f832d9bbc, group_id=a20ac3b450304e798825c539367360f0, role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:04.928594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.929640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b80918-d248-40bb-902c-9eee69d7453d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:04.930243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 213/413] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:04 2022] GET /identity/v3/projects/34b2b6cdb81943ad9b9237874eeca93b => generated 344 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:04.935874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:04.964086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 201/414] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 10:52:04 2022] PUT /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc/groups/a20ac3b450304e798825c539367360f0/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:05.058501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c9566-8fb6-4922-8f4e-49413af77db6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.059130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c9566-8fb6-4922-8f4e-49413af77db6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.059130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c9566-8fb6-4922-8f4e-49413af77db6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.061853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 202/415] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:05.067481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.067767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.068074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.097884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:05.099132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:05.135793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136d650-c268-43c3-9c48-92fc1953c4db None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:05.163208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:05.202759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:05.203544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.203854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.204413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] PATH_INFO: `/v3/projects/34b2b6cdb81943ad9b9237874eeca93b/groups/fa42ce4197764dde835c642966f818a9/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.207228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] RBAC: Authorizing `identity:create_grant(project_id=34b2b6cdb81943ad9b9237874eeca93b, group_id=fa42ce4197764dde835c642966f818a9, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:05.208717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:05.213286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 203/416] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:52:05.225230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 214/417] 10.222.0.20 () {58 vars in 1442 bytes} [Wed Aug 31 10:52:04 2022] PUT /identity/v3/projects/34b2b6cdb81943ad9b9237874eeca93b/groups/fa42ce4197764dde835c642966f818a9/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:52:05.497683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:05.615948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ec9ff-b96b-4e5d-850a-06a967100bc0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:05.666256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:05.666949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.667261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.667573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.685265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:05.690373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:05.691274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:05.692546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 204/418] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:05.699371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:05.720569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:05.857127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db954f8-c164-4eaf-bb64-b207a2834de9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:05.895381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:05.895991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.896299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.896573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.897243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] RBAC: Adding query filter params (name=member) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:05.897548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:05.898930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ae9c1-7609-4b86-ab67-ac99e44427da None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:05.907951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 215/419] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:52:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:05.913757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:05.949536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750a77-42d8-4fda-b3de-6271a46e0602 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.949848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750a77-42d8-4fda-b3de-6271a46e0602 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.950100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14750a77-42d8-4fda-b3de-6271a46e0602 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.951605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 216/420] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:05.960146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:05.960432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:05.960713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:05.962513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:05.991949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:05.993180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:06.065452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76e40a-ea85-4526-a1da-498acf07a3aa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:06.126513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:06.127195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:06.127481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:06.127760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:06.132505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:06.150430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 217/421] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:52:06.207175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:06.208688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:06.244754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:06.351942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 205/422] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:06.361231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:06.486658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:06.507745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabf2b6-a085-4538-80d3-153f12d667e7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:06.557836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d9d4c-17ab-479b-b259-28b765e25507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:06.558541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9d4c-17ab-479b-b259-28b765e25507 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:06.558848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9d4c-17ab-479b-b259-28b765e25507 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:06.559153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9d4c-17ab-479b-b259-28b765e25507 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:06.561289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9d4c-17ab-479b-b259-28b765e25507 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:06.563830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9d4c-17ab-479b-b259-28b765e25507 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:06.567167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 218/423] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:06 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:06.568476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ee1df-cc65-4da4-a009-9865e70be774 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:06.568923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ee1df-cc65-4da4-a009-9865e70be774 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:06.569113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ee1df-cc65-4da4-a009-9865e70be774 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:06.570684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 219/424] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:06.576866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ee1df-cc65-4da4-a009-9865e70be774 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:06.607378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2fca6-05c6-49b6-bf3d-dd6d017c28e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:06.644095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19dbcf-b032-407c-b2e0-92889822cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:06.644885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19dbcf-b032-407c-b2e0-92889822cd74 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:06.645260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19dbcf-b032-407c-b2e0-92889822cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:06.645633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19dbcf-b032-407c-b2e0-92889822cd74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:06.647974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:06.660054 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19dbcf-b032-407c-b2e0-92889822cd74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-dd19dbcf-b032-407c-b2e0-92889822cd74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:06.665877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:06.667964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 10:52:06.670035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:06.676044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 206/425] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:06.676044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:06.676044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:06.676044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:06.684160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:06.700851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:06.700851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:06.747768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ee1df-cc65-4da4-a009-9865e70be774 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:06.805764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:06.806911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:06.807241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:06.807563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:06.809119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:06.811033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:06.812272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 207/426] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:06.813105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 220/427] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:06 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:06.813885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:06.820733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:06.968528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9527afa-b19d-445f-92c8-cab7c9142a25 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:06.976087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9832f5-23c6-4933-8316-6fed26ef7201 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:07.001123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:07.017459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.018352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.018640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.019826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.019826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] RBAC: Adding query filter params (name=member) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:07.020638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:07.022606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:07.030489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.031178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.031496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.031836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.033203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=8e17314c11454ebc92e55967eee83554) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:07.033589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:07.039886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:07.043829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 208/428] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:52:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:07.044181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:07.053253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:07.057251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 221/429] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:06 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:07.219297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8336f85-bff5-4208-b4eb-c0b49dd12459 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:07.266186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8683aaf9-9a1f-4e3a-af1b-34ffcc5a194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.266930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683aaf9-9a1f-4e3a-af1b-34ffcc5a194d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.267242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683aaf9-9a1f-4e3a-af1b-34ffcc5a194d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.267546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8683aaf9-9a1f-4e3a-af1b-34ffcc5a194d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.272268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:07.349456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:07.372584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683aaf9-9a1f-4e3a-af1b-34ffcc5a194d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:07.374454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8683aaf9-9a1f-4e3a-af1b-34ffcc5a194d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:07.451482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1565d9-677d-432e-8c22-aa2c4508ed31 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:07.511450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11afe8-506e-4a47-a311-fecceea410bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.512120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11afe8-506e-4a47-a311-fecceea410bc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.512398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11afe8-506e-4a47-a311-fecceea410bc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.513882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11afe8-506e-4a47-a311-fecceea410bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.531593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11afe8-506e-4a47-a311-fecceea410bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-8e11afe8-506e-4a47-a311-fecceea410bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:07.538374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:07.539308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:07.540454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 222/430] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:07.541343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed61a6-c009-413a-a8f1-de64c8db8721 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.541343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed61a6-c009-413a-a8f1-de64c8db8721 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.541343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ed61a6-c009-413a-a8f1-de64c8db8721 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.542740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 223/431] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:07.545828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 209/432] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:07.547556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed61a6-c009-413a-a8f1-de64c8db8721 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:07.548801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.549090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.549367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.571192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:07.572850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:07.641502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:07.659180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 210/433] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:07.659917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:07.711469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ed61a6-c009-413a-a8f1-de64c8db8721 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:07.716679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:07.756924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.757569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.757843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.758125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.758852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:07.759205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:07.760913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:07.774577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 224/434] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:07.781809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:07.852336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec458b6-a0bf-434d-aa01-1bdacbbe450e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:07.900319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.900950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.901225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.901492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:07.902608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:07.904076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:07.906133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 211/435] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:07 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:07.908976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a712e-7e87-4e60-ad4c-648c7e2b846e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:07.911291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:07.937425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:07.953646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:07.954510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:07.954876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:07.955205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.005794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.031026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b217b12-c530-4a96-bd2a-9af483d97a12 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.038585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.041141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.070823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662a77-4b77-422a-826d-090a1b75aa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.071540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662a77-4b77-422a-826d-090a1b75aa30 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.071822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662a77-4b77-422a-826d-090a1b75aa30 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.072126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662a77-4b77-422a-826d-090a1b75aa30 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.073246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f662a77-4b77-422a-826d-090a1b75aa30 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f662a77-4b77-422a-826d-090a1b75aa30 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.076327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 212/436] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:07 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:08.077442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac386-ec64-4472-a905-adab2eea7695 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.078341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac386-ec64-4472-a905-adab2eea7695 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.078341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aac386-ec64-4472-a905-adab2eea7695 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.083562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 213/437] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:08.083562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac386-ec64-4472-a905-adab2eea7695 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.149163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 225/438] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:08.152951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.176678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.188954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aac386-ec64-4472-a905-adab2eea7695 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.223288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249fc021-aa64-472d-90c3-cc1c2b8397f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.223986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fc021-aa64-472d-90c3-cc1c2b8397f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.224201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fc021-aa64-472d-90c3-cc1c2b8397f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.224468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249fc021-aa64-472d-90c3-cc1c2b8397f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.236572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249fc021-aa64-472d-90c3-cc1c2b8397f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-249fc021-aa64-472d-90c3-cc1c2b8397f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:08.241402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:08.242360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:08.243215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 214/439] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:08.243942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.243942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.244664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.259203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ddff41-7c03-4715-9074-63b5a4e9d60f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.261798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:08.263077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:08.270647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.294270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.294798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.295080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.295357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.296112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=8249b61571f3472089a8d3cc13b0c9df, scope.project.id=9ff6481d416344218c56ae7f832d9bbc) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:08.296429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.298409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.304162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 226/440] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=8249b61571f3472089a8d3cc13b0c9df => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:08.304951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.349558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 215/441] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:08.350461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.407832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.424970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98f51f-7855-45e4-a761-663bdbe4696a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.467296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.467941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.468312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.468590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] PATH_INFO: `/v3/projects/8e17314c11454ebc92e55967eee83554` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.471222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] RBAC: Authorizing `identity:get_project(project_id=8e17314c11454ebc92e55967eee83554)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.472766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.479834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 227/442] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/projects/8e17314c11454ebc92e55967eee83554 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:08.485767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.489569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6fe6b-7e12-4f33-b257-fbfcac089d59 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.541468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.542117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.542401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.542681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.543365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:08.543672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.545411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.555162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 216/443] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:08.560895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.564711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.613854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fc100-df8b-46f6-a07c-9f1bc523ca66 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.649290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.649916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.650194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.650464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.651560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.653005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.654557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 228/444] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:08.655307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.670894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed1fea5-141b-421d-81f2-0b04f5e9eb8a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.677644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.706968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.707672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.708060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.708393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.760677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ccb0ef-b2d7-4a68-bad3-f9c3c2566a28 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.778707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.780221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.784952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.799911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.800700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.800996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.801257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.813033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:08.818367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:08.819286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:08.820103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 229/445] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:08.821376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.887161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 217/446] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:52:08.889110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.903464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:08.923350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f649ee-7d7f-4f79-8f8e-1f0967ace7a2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:08.958235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:08.958869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:08.959150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:08.959424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:08.960187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=8e17314c11454ebc92e55967eee83554) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:08.960500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:08.961927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:08.967899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 230/447] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=8e17314c11454ebc92e55967eee83554&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:08.968706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:08.999650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd580f94-3835-4763-8781-4a450ee65a17 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.003842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:09.044211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.044997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.045275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.045541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.046250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:09.046549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.048041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.057618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 218/448] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:09.064264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:09.101480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8136de2-e20a-4477-a7c1-8d5a42334ef2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.146634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.147830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.147830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.147830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] PATH_INFO: `/v3/projects/9ff6481d416344218c56ae7f832d9bbc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.150372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] RBAC: Authorizing `identity:get_project(project_id=9ff6481d416344218c56ae7f832d9bbc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.150493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.152279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 231/449] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:08 2022] GET /identity/v3/projects/9ff6481d416344218c56ae7f832d9bbc => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:09.158471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:09.178183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:09.191026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afccda-a53a-484b-b96d-eba596c47ae6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.234220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.234828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.235647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.236176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.283222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:09.294982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577a36e0-97c6-4ac3-9918-3a4c7131231e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.329276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.331280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.345542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.347161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.347457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.347769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] PATH_INFO: `/v3/groups/a20ac3b450304e798825c539367360f0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.350258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] RBAC: Authorizing `identity:get_group(group_id=a20ac3b450304e798825c539367360f0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.351681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.353374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 232/450] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:09 2022] GET /identity/v3/groups/a20ac3b450304e798825c539367360f0 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:09.361052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:09.464038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 219/451] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:52:09.468055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:09.477525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:09.503966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47beaa22-e547-4df6-8b4a-fad7a0ec0552 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.537003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.537601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.537876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.538146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.538891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] RBAC: Adding query filter params (group.id=a20ac3b450304e798825c539367360f0, role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=9ff6481d416344218c56ae7f832d9bbc) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:09.539197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.540658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.546385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 233/452] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:09 2022] GET /identity/v3/role_assignments?group.id=a20ac3b450304e798825c539367360f0&scope.project.id=9ff6481d416344218c56ae7f832d9bbc&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:09.549135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:09.590531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c6546-07a3-44f5-8c54-ba8826c922c7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.628345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.628987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.629271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.629549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] PATH_INFO: `/v3/projects/34b2b6cdb81943ad9b9237874eeca93b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.630727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] RBAC: Authorizing `identity:get_project(project_id=34b2b6cdb81943ad9b9237874eeca93b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.632241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.634180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 220/453] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:52:09 2022] GET /identity/v3/projects/34b2b6cdb81943ad9b9237874eeca93b => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:09.639717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:09.747970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d9656-1d21-47de-8a6c-d9936cb0edf2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.785476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc120d7-efd2-43c3-8ccb-f8998002a12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.785944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc120d7-efd2-43c3-8ccb-f8998002a12c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.786223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc120d7-efd2-43c3-8ccb-f8998002a12c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.786497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc120d7-efd2-43c3-8ccb-f8998002a12c None admin] PATH_INFO: `/v3/groups/fa42ce4197764dde835c642966f818a9` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.787615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc120d7-efd2-43c3-8ccb-f8998002a12c None admin] RBAC: Authorizing `identity:get_group(group_id=fa42ce4197764dde835c642966f818a9)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.789073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc120d7-efd2-43c3-8ccb-f8998002a12c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.790690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 234/454] 10.222.0.20 () {56 vars in 1180 bytes} [Wed Aug 31 10:52:09 2022] GET /identity/v3/groups/fa42ce4197764dde835c642966f818a9 => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:09.796744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:09.909736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce9692-4112-42d9-b2c3-027fffd5d0f9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:09.942410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:09.943000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:09.943277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:09.943548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:09.944296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] RBAC: Adding query filter params (group.id=fa42ce4197764dde835c642966f818a9, role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=34b2b6cdb81943ad9b9237874eeca93b) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:09.944604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:09.946024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5055cda5-8f10-4754-88de-9d37fe66215a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:09.947720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:09.952649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 221/455] 10.222.0.20 () {56 vars in 1376 bytes} [Wed Aug 31 10:52:09 2022] GET /identity/v3/role_assignments?group.id=fa42ce4197764dde835c642966f818a9&scope.project.id=34b2b6cdb81943ad9b9237874eeca93b&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:12.244303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f4514-114d-47db-959c-f83bb39d53b3 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:12.244814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f4514-114d-47db-959c-f83bb39d53b3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:12.244915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f4514-114d-47db-959c-f83bb39d53b3 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:12.246176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 235/456] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:12.252388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806984-3c00-42ab-a8bc-d70cf7d4b66b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:12.252835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806984-3c00-42ab-a8bc-d70cf7d4b66b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:12.253009 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72806984-3c00-42ab-a8bc-d70cf7d4b66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:12.282649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-72806984-3c00-42ab-a8bc-d70cf7d4b66b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:12.284396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-72806984-3c00-42ab-a8bc-d70cf7d4b66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:12.395304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 222/457] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:12.635949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c18448-182a-48f6-8c6f-b176ef09772b None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:12.635949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c18448-182a-48f6-8c6f-b176ef09772b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:12.635949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c18448-182a-48f6-8c6f-b176ef09772b None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:12.635949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 236/458] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:12.642092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:12.642450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:12.642765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:12.669020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:12.670241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:12.750974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c506f-986c-4784-9309-ca0b3bc572c3 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:12.750974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c506f-986c-4784-9309-ca0b3bc572c3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:12.750974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c506f-986c-4784-9309-ca0b3bc572c3 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:12.751776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 237/459] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:12.756891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:12.757424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:12.757914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:12.770328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 223/460] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:12.783124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:12.784403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:12.798603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:12.879777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:12.896327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 238/461] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:12.960404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:13.018666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c97872-f353-43e3-aad2-eeb8ff4d0786 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.072476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.073120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.073402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.073749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.096173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.097976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:13.119910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:13.120870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:13.121710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:13.121710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:13.121710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:13.123483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 224/462] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:13.131413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.165180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.304738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab043cc4-4593-4382-9653-80a37f9a4f75 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.305400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54446e0e-ee00-47a0-bf85-e8da2daea86f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.318333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:13.346639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.347266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.347585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.347804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.349639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.350442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.350880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.351155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.351805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:13.352304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.353819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e18f7c-417c-4cab-9e04-5386c063e709 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.360654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.362123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.367441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 239/463] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:13.369106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62dcf-d89e-47ce-a842-26f67ccfca5d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.369464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62dcf-d89e-47ce-a842-26f67ccfca5d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.369742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d62dcf-d89e-47ce-a842-26f67ccfca5d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.371141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 240/464] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:13.372203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62dcf-d89e-47ce-a842-26f67ccfca5d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.378169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:13.384576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:13.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:13.386678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:13.386678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:13.386678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:13.386678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 225/465] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:13.387002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.504382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be5ccf-4cd6-48f4-aa1f-12e0068b7b98 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.509067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d62dcf-d89e-47ce-a842-26f67ccfca5d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.546509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d64bb6e-0e74-4bac-a9b6-752ada9e729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.547624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bb6e-0e74-4bac-a9b6-752ada9e729c None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.548691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bb6e-0e74-4bac-a9b6-752ada9e729c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.549148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.549380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d64bb6e-0e74-4bac-a9b6-752ada9e729c None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.549787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.550070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.550356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.550833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64bb6e-0e74-4bac-a9b6-752ada9e729c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.552521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d64bb6e-0e74-4bac-a9b6-752ada9e729c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.564452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.566240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.580590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 226/466] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:52:13 2022] POST /identity/v3/users => generated 246 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:13.582159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.582580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.582993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:13.584956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:13.585865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:13.586705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:13.586705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:13.586705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:13.586705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 241/467] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:13.586982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.607907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:13.609169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:13.626932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:13.676637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:13.704370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 227/468] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:13.705608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.770648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995820d-2150-444a-9eae-ed165e34c09d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.829187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.830356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.831194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.831914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.832765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:13.833266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.835744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.853569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 242/469] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:13.859610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:13.886125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352df745-7570-4b2f-b305-58f8f27d4c1c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:13.940063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:13.940672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.942661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.942661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.942661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:13.942661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:13.943519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b04d62-3608-4900-9973-cb18c4bbea30 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:13.971571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 228/470] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:13.973269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7a226-cff2-4aee-ab8f-fb774298f77b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.973759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7a226-cff2-4aee-ab8f-fb774298f77b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.974278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7a226-cff2-4aee-ab8f-fb774298f77b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:13.975991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 229/471] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:13.981236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:13.981787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:13.982259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.002145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:14.003663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:14.005227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1ba8d0-2069-425a-99ee-f07095085432 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.006357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:14.021255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:14.042522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.043132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.043409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.043681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.044370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:14.045809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:14.068448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 243/472] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:52:13 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:14.069252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.091787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 230/473] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:52:14.093079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.280081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e6ef4c-2681-495e-9394-b1e506e9b02c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.280270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3479095d-f938-4fd4-b5ea-8e5663287ccd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.323142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.323916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.324338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.324706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.325457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:14.326978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:14.331751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.332915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.333336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.333770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.343402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:14.348876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:14.351292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:14.356546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 244/474] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:52:14 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:14.372456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:14.373441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:14.374301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:14.374301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:14.374301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.374301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 231/475] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:14.379674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.391965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.413850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:14.514624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d580b1-9557-4d79-851c-10df326f0281 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.522839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df159040-dd5f-41dc-b9d2-382063eb194b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.559019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.559554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.559835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.560143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.560988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:14.561323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:14.563761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:14.569474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.570382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.570868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.571346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.572901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:14.575339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:14.577432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 245/476] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:14.584286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.sql.core [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 10:52:14.584286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 10:52:14.584286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [parameters: {'id': 'd454e2ad8cf94ce1836386e1de62e1bd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 10:52:14.584286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 10:52:14.584646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 31 10:52:14.606621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:52:14.607561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application yield resource Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise self.value Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application session.commit() Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 10:52:14.608530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.flush() Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application compat.raise_( Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 10:52:14.609390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:52:14.610474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:52:14.611277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [parameters: {'id': 'd454e2ad8cf94ce1836386e1de62e1bd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:14.617408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 10:52:14.618522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.619524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 232/477] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 10:52:14 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:14.621096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.688450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:14.700056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:14.726783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd79ed-c717-44f9-8fd5-ec612449af46 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.744131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ca287-a2e8-4dc0-9bbe-769fb36fc485 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.791708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c885b79-0325-4213-a746-df033e771ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.793005 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c885b79-0325-4213-a746-df033e771ada None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.793782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c885b79-0325-4213-a746-df033e771ada None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.794683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c885b79-0325-4213-a746-df033e771ada None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.795444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb58622f-19fa-45e9-8033-f557d3944db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:14.796244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58622f-19fa-45e9-8033-f557d3944db4 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:14.796710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58622f-19fa-45e9-8033-f557d3944db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:14.797152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb58622f-19fa-45e9-8033-f557d3944db4 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:14.798076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58622f-19fa-45e9-8033-f557d3944db4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:14.799760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb58622f-19fa-45e9-8033-f557d3944db4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:14.807390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c885b79-0325-4213-a746-df033e771ada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0c885b79-0325-4213-a746-df033e771ada None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:14.812935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:14.814052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:14.814849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 233/478] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:14.825574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c885b79-0325-4213-a746-df033e771ada None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:14.841361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 246/479] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:52:14 2022] POST /identity/v3/users => generated 249 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:14.968271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c885b79-0325-4213-a746-df033e771ada None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:14.985767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:15.011564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:15.012619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:15.013124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:15.013621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:15.014545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:15.015088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:15.017379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75187a0d-8e58-43e1-9b73-e2435649dadf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:15.031595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 234/480] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:16.072816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ddfce-8443-4289-af2e-0b17e005a177 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:16.073306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ddfce-8443-4289-af2e-0b17e005a177 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:16.073306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ddfce-8443-4289-af2e-0b17e005a177 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:16.074857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 247/481] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:16.090148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d5546-bb68-4858-a1a9-719f73809e1c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:16.090620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d5546-bb68-4858-a1a9-719f73809e1c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:16.090694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d5546-bb68-4858-a1a9-719f73809e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:16.116963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a47d5546-bb68-4858-a1a9-719f73809e1c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:16.118143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a47d5546-bb68-4858-a1a9-719f73809e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:16.206214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 235/482] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:16.599119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ddfce-8443-4289-af2e-0b17e005a177 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:16.693777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9957ad-8e93-4768-b1da-40f280d30a32 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:16.694419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9957ad-8e93-4768-b1da-40f280d30a32 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:16.694955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9957ad-8e93-4768-b1da-40f280d30a32 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:16.696832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 236/483] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:16.702215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:16.702746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:16.703237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:16.741309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:16.742529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:16.747502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ddfce-8443-4289-af2e-0b17e005a177 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:16.811305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:16.811963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:16.812257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:16.812533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:16.842123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:16.847644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:16.848571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:16.849403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:16.852640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 248/484] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:16.859775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:16.863745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:16.884684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 237/485] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:16.986632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29582880-20d8-416f-ac5e-90db1211fb0b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.022857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.023477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.023748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.024030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.024707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:17.025008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:17.026383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c14312-54b2-4604-ba02-80fdd1b5c657 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:17.035768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 249/486] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:17.042781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.069915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67869-38a6-4a8d-b633-9d4205e02e34 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.070242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67869-38a6-4a8d-b633-9d4205e02e34 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.070460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e67869-38a6-4a8d-b633-9d4205e02e34 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.071989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 250/487] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:17.080838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.082302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.082804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.094993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.113253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:17.114565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:17.178854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.197656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 251/488] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:17.214453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.217362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931321c-f34f-45ea-952c-64e2451fb5c5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.263961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.264636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.265344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.266690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.279769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:17.281263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:17.297926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:17.298871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:17.299791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:17.299791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:17.299791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:17.299966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 238/489] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:17.305760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.433691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f8844-fa60-4581-91ad-42b70b3f8ca4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.442291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.463953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81962b9-dd4a-42ac-8530-de1580e0942d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.494173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.494808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.495089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.495364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.507039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:17.510994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.511604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.511886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.512198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:17.512461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:17.513315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:17.514130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:17.514743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:17.514743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:17.514743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:17.518672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 252/490] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:17.519555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.529671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 239/491] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:17.530590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.588747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.636085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60700a7-0700-447b-a9f4-758a2275dc7f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.644163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bc5-2f5b-4fa7-bf35-cd3d7f2b2b76 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.670610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.671222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.671498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.671767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.678628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.679237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.679732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.679842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.680475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:17.680783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:17.682172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8e3ce-049f-4268-bb8d-6d050b28fb8d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:17.682277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:17.687026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:17.688888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:17.689679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 253/492] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:17.690350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.694988 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 240/493] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:17.696200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bb5f1-4640-40a7-a9ed-5f939d7220ed None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.696493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bb5f1-4640-40a7-a9ed-5f939d7220ed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.696769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078bb5f1-4640-40a7-a9ed-5f939d7220ed None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.698265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 241/494] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:17.703859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc7c20-c973-4fd8-8013-1d9e79282d8d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.704307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc7c20-c973-4fd8-8013-1d9e79282d8d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.704866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc7c20-c973-4fd8-8013-1d9e79282d8d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.706202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 242/495] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:17.706923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc7c20-c973-4fd8-8013-1d9e79282d8d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.721201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.795650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb79157-e3f1-429e-bc8d-69e121ae28b7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.810366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.813402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc7c20-c973-4fd8-8013-1d9e79282d8d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.831320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.831944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.832257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.832529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.839601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:17.841317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:17.848146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:17.848750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.849033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.849308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.849969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:17.850275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:17.851643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e1e62-1f95-4aea-8ab2-b45e4c01b135 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:17.854550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:17.855473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:17.856381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 254/496] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:17.860537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:17.861947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 243/497] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:17.863022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e064752-e6e2-4795-a54c-e1e15d79a361 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.863314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e064752-e6e2-4795-a54c-e1e15d79a361 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.863590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e064752-e6e2-4795-a54c-e1e15d79a361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.881570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9e064752-e6e2-4795-a54c-e1e15d79a361 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:17.882781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e064752-e6e2-4795-a54c-e1e15d79a361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:17.952202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 244/498] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:17.953451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:17.953744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:17.954275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:17.960560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:17.974780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:17.975316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68c00e-9dfd-4b81-9aa6-744e89c5d8bc None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:17.975976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:18.011476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.012246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.012606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.012889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.024691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.026160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:18.041678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:18.042628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:18.043463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:18.043463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:18.043463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:18.043463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 255/499] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:18.043687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.060215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 245/500] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:18.060215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.076459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.157881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c8ae2-64a5-4739-957b-03168beb99eb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.196277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.197163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.197450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.197729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.198921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:18.199482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.201268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4fdce-6500-449a-ba1a-017ebbac8c9e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.201711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.236931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.242223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.243271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.243553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.243832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.252096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 256/501] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:18.252096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.255015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.256044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:18.271595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:18.272723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:18.273573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:18.273573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:18.273573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:18.273573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 246/502] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:18.273791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.343277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.360293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a643de-28f5-4f75-b1d3-20624112b7b6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.380666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd871dc-b265-42b3-95bb-b3a3cd36077c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.394620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.395218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.395495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.395768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.396436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:18.396739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.398102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.409755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 257/503] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:18.410554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.414909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.415539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.415820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.416137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.427193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.428675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:18.442974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:18.443859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:18.444750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:18.444750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:18.444750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:18.444750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 247/504] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:18.444961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.483023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.512677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4637247-49a3-4676-8500-3af9df64f3d8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.546291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.546912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.547196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.547470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.548664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43a637-fddd-4a5c-8225-f0cd490ddea0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.561820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.583431 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.584057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.584344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.584622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.585266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:18.585575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.586966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.599890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 248/505] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:18.600734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.616266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.617715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.694261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.700443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49675fdf-d66e-4637-8fbb-391c6a3b2cb8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.715468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 258/506] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:18.716736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.735930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.736685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.736947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.737216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.748940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:18.755316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:18.756425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 249/507] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:18.758349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.769656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:18.829731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae85121-0495-4e81-b0d6-b2ffe0aed4dd None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.866661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.867303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.867584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.867858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.872259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7174e1-ed30-4f8f-a9fc-2ea30e3ee174 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:18.876255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.877732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:18.891228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:18.892439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 259/508] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:18.893479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.913554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:18.914179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:18.914459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:18.914741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:18.915424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:18.915719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:18.917182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:18.929742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 250/509] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:18.930633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:18.951246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.008329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680d8a9-8239-4500-a5d2-a4b5e54709bb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.010836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.042939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c33b6-4bf5-4f76-bd06-80d867f36884 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.045139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.045845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.046132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.046407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.055963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.057456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:19.070115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:19.071029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:19.071929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 260/510] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:19.072523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.083098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.083733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.084093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.084330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.095152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.096633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:19.111427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:19.112471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:19.113471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:19.113471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:19.113471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:19.113471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 251/511] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:19.113471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.173414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c08d8e-063a-4ca9-8900-20b70fb2eeaa None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.182223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.209084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.209690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.209981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.210256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.210921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:19.211231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.212698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.217820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54b5c4-200f-49f1-9be8-a20b3de48d4a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.221657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 261/512] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:19.222482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.256030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.256656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.256946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.257205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.257946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:19.258253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.258769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.260068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.299312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 252/513] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:19.300141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.339542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be39b0-1fe8-4504-a40a-9948665bbb28 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.375260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.375886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.376189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.376462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.377126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.378565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.400795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 262/514] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:52:19 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:19.401623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.403969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65401e31-9569-4972-a1ed-a60ed319a7ab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.440402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.453302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.453929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.454208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.454481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.455341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:19.455653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.457230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.478573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 253/515] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:19.479906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.515107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.538134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c79cbb-d96d-4c78-8c83-ce24736c46a4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.580908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.581749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.582170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.582571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.583359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:19.583814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.586078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.601317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 263/516] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:19.602470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.605562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc200368-ac06-4ccb-9b9d-650d2ed1c26d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.641357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.641987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.642262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.642533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.652849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.654307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:19.669485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:19.670421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:19.671307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:19.671307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:19.671307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:19.671307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 254/517] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:19.671529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:19.711030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.719347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc611d4-d25c-4e61-999b-f36cfdcefdd1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.760210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.761178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.761459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.761723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.776408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.791218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823aa55b-b655-4585-82f0-6c3f07ea1224 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:19.827663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:19.828444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:19.828728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:19.829010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:19.839576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.841251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.903491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:19.905011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:19.946449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:19.949136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 264/518] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:19.949993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.004817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.024549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 255/519] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:20.025656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.090919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686f3e0-e223-4353-b667-d70c132ad373 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.153203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.154456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.154740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.155019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.162918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.164475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.172627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7352d7c8-7b9c-4818-ab0e-dd17716b4353 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:20.178077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:20.179251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 265/520] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:20.180121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.216247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.216849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.217130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.217400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.218047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:20.218572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.220258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.223014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.239156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 256/521] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:20.239156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.282195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a74fc-c9fa-4c9b-bc83-edd4b1770770 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.283920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.325974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.326595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.326872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.327144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.337249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.338678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.346971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319eab5-1ef9-48b9-851a-d55905b89d2f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:20.358696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:20.360391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:20.361246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:20.361246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:20.361246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:20.361246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 266/522] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:20.362234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.387365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.387992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.388291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.388563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.400868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.402373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:20.419375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:20.420531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:20.421709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:20.421709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:20.421709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:20.421709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 257/523] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:20.421709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.485805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8706-68d2-4b1a-8806-46fa93a46176 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.486170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.526766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.527406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.527688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.530298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.534030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:20.534357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.535849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.554967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cff3b6-cbc8-4b68-a046-82aab19c9218 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.555193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 267/524] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:20.555906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.576510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.598230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.598880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.599170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.599446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.607748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.609775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:20.628024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:20.629026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 258/525] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:20.629982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.681112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153b70ac-2b81-4f68-a777-4bdc45c12ece None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.719571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.720461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.720741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.721019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.721660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:20.721964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.723338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.740101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 268/526] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:20.740101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.745136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea35573-f8b2-4fec-bed0-937b80e3d6cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.755248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.793302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.794462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.794751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.795029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.795692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:20.796407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.798755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.817805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 259/527] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:20.817805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.846154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:20.881418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a23b4a3-38ec-483b-b42a-3cd9b6f191d5 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:20.926268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:20.927013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:20.927761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:20.927761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:20.928709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=508f557ac5e34c67a31e4d55f8142a0f) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:20.929093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:20.930662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:20.948045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 269/528] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/role_assignments?user.id=508f557ac5e34c67a31e4d55f8142a0f&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:20.948045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:20.972114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3abb4-fd9a-42f4-8d46-75ffee9ed083 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.016434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.017345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.017739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.018114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.019123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:21.019507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.021553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.044260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 260/529] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:21.045257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.083328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:21.107424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a97ac-f6ec-4ff3-9582-2cea942af450 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.149926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.150549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.150834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.151211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.160545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.162087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.169469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:21.169769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64d063-2651-4299-be82-26ffde0fbfe1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:21.177536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:21.179012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 270/530] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:21.179982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.210750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.211500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.211783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.212084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.221445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.223369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:21.241657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:21.242911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:21.243783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:21.248288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 261/531] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:21.249279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.331874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afb015-a4df-4c5e-af27-4730dac9ffe6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.352722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:21.371049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d1eaf-f7f1-44c5-a84a-89b39f174c82 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.392824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.394544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.395057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.395964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.410497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.411110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.411395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.411668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.412423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:21.412727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.414185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.442389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 262/532] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:21.443584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.489165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:21.492745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.495798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.600409 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6768f4-f32a-4f88-91d1-f866a15f90df None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.612048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 271/533] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:21.612048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.650983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.651691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.652140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.652558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.653653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:21.654208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.657496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76af039-86af-4717-8202-1f404e83a7db None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.673452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 263/534] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:21.675060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f544be5-066b-4d13-aaac-8f2b035572b6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.675352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f544be5-066b-4d13-aaac-8f2b035572b6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.675663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f544be5-066b-4d13-aaac-8f2b035572b6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.677322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 264/535] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:21.679140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f544be5-066b-4d13-aaac-8f2b035572b6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.693592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:21.740933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:21.745450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a8f82-2fdb-414e-8a43-54b104c7cbdf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.788761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.789742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.790178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.790583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.791459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=72251fea0df54206b554236c14296a7b) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:21.792082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.793870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.801241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 272/536] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/role_assignments?user.id=72251fea0df54206b554236c14296a7b&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:21.802281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:21.834630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f544be5-066b-4d13-aaac-8f2b035572b6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:21.885869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:21.886777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.887205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.887623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.901283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:21.902929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-89bc3eae-f64f-4aca-87e7-d02fb0ff4028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:21.930302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:21.931238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:21.932817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:21.932817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:21.932817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:21.933536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 265/537] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:21.934845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:21.935490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:21.935490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:21.958730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:21.960352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:21.961588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8790784-64f3-417c-9718-945b891597f4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:22.007498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:22.008506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:22.008993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:22.009572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:22.010765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=e1aaf56b3e654c0bb65d0c804fd797e6) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:22.011280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:22.013112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:22.020238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 273/538] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:21 2022] GET /identity/v3/role_assignments?user.id=e1aaf56b3e654c0bb65d0c804fd797e6&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:22.021421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:22.042334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:22.076418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:22.078960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 266/539] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:22.168128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3014d4-932e-4679-96f0-bf5f009056e1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:22.210248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:22.210892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:22.211167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:22.211330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:22.212031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:22.212340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:22.213708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:22.232226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 274/540] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:22.238451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:22.382382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68520eb-e8f7-4b3a-ad4e-6a83f4b6b59d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:22.427307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:22.427945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:22.428251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:22.428525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:22.441505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:22.442950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:22.457788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:22.458667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:22.460046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 267/541] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:22.464629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:22.577167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:22.602783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e94e4b-e0cf-4dad-b637-ce18a4b76025 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:22.650966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:22.651565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:22.651843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:22.652139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:22.652827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:22.653135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:22.654561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:22.670733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 275/542] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:22.676744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:22.724064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:22.754526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:22.845589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1634e5-d791-44b5-bb06-d279f9d52bb7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:22.866593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46e407-2c4a-4aae-a669-3a545f825e1f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:22.882997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:22.883981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:22.884294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:22.884571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:22.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:22.902559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:22.903442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:22.904281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:22.908246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 276/543] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:22.910604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:22.932367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:22.984844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:22.985387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:22.985672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:22.985947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:22.986685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=49f7536740a54bdbba5b6c81f732c439) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:22.986994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:22.988491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:23.014019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 268/544] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:22 2022] GET /identity/v3/role_assignments?user.id=49f7536740a54bdbba5b6c81f732c439&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:23.024619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639106c1-dbb5-4cab-9535-e8e470512c6d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:23.063130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:23.063754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.064062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.064353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.065015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:23.065316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:23.066693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28272273-4215-4fc2-9031-7f9ae6f9695b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:23.075465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 277/545] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:23.082047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:23.110838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2ff1-391c-467c-9314-6227f860134e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.111159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2ff1-391c-467c-9314-6227f860134e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.111553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff2ff1-391c-467c-9314-6227f860134e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.113469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 278/546] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:23.119393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.119766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.120159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.139498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:23.141424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:23.218740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c557d5-46b5-4d98-ba05-99b7b1dd44ab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:23.236509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:23.261219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:23.262758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.263410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.263901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.278666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:23.281092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:23.303433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:23.304803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:23.305766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:23.305766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:23.305766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:23.312122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 269/547] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:23.315550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:23.350525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:23.371134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 279/548] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:23.520501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647a9d1-9838-41c2-bed1-b5274d327364 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:23.574991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:23.575623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.575910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.576243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.576937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:23.577329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:23.578848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:23.596203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 270/549] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:23.602941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:23.652039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:23.701464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:23.785387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9618bab2-9358-4958-8cb1-06bd6bc389ab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:23.789684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc77c2d-7fe2-4546-bff0-7542fc0741df None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:23.836223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:23.837246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.838915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.839391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.851909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:23.852896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:23.853373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:23.854026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:23.870468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:23.873448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:23.880152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:23.881246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:23.882051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:23.883712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:23.893380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 271/550] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:23.898999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:23.902040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:23.902988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:23.904199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:23.919809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 280/551] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:23.923290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:24.043054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:24.097942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:24.122258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed4e7-6301-4a8c-b35d-a3068da3e83d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:24.125950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a750db4-f2cd-41d6-9534-95a4c8348ec9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:24.176731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:24.176970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:24.177340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.177622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.177871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.177973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.178343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.178660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:24.178804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.178990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:24.179615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:24.180130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:24.180498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:24.181953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:24.201820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 281/552] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 10:52:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:24.208986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:24.210586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 272/553] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:24.217208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:24.361660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2481a8f2-8ebf-4304-9760-4982602d2728 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:24.372768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86daa8-2032-45c9-982f-c516e81320ce None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:24.386220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:24.410311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04d266d-c0ed-4ec2-9122-4d3c37489602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:24.411902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d266d-c0ed-4ec2-9122-4d3c37489602 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.412428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d266d-c0ed-4ec2-9122-4d3c37489602 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.412905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04d266d-c0ed-4ec2-9122-4d3c37489602 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.440652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:24.442312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.442781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.443334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.474303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:24.477724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:24.501044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5af822ef-38b2-4f14-8fb1-45cbe7ea6e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:24.511070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:24.512191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:24.513199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:24.513199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:24.513199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:24.513199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 273/554] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:24.514031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2192c3-a40a-4776-afb9-2833405c19f3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.514132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2192c3-a40a-4776-afb9-2833405c19f3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.514504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2192c3-a40a-4776-afb9-2833405c19f3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.515749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d266d-c0ed-4ec2-9122-4d3c37489602 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:24.516448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 274/555] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:24.518069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04d266d-c0ed-4ec2-9122-4d3c37489602 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:24.520460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2192c3-a40a-4776-afb9-2833405c19f3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:24.638707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2192c3-a40a-4776-afb9-2833405c19f3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:24.649376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 282/556] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:52:24.650769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e4bbb-f79d-4cae-bfea-179884ee9ee0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.650980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e4bbb-f79d-4cae-bfea-179884ee9ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.651211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e4bbb-f79d-4cae-bfea-179884ee9ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.672056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e48e4bbb-f79d-4cae-bfea-179884ee9ee0 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:24.673231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e48e4bbb-f79d-4cae-bfea-179884ee9ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:24.680727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:24.681739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.681739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.681939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.682557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:24.682864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:24.684264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:24.685672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:24.699151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 275/557] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:24.699989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:24.756105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 283/558] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:24.757868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c03a2-e629-42c3-b492-265ced37ec50 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.758377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c03a2-e629-42c3-b492-265ced37ec50 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c03a2-e629-42c3-b492-265ced37ec50 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.764950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 284/559] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:24.766411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c03a2-e629-42c3-b492-265ced37ec50 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:24.783033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:24.831696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927d8d3-036f-4e8d-b10b-f4da456e39f5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:24.869389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:24.870081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.870355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.870623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.881608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:24.883076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cf6d0c39-31b3-457e-9cb9-fff7e6bd1515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:24.898328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:24.899435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:24.901100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:24.901100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:24.901100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:24.903960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 276/560] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:24.905258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:24.905543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:24.905814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:24.925326 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:24.926504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:24.950155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c03a2-e629-42c3-b492-265ced37ec50 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.014578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 277/561] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:25.014578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.014938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.014938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.014938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.014938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.036114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.037919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:25.060036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:25.061532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:25.064217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 285/562] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:52:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:25.069296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.087964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.098304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.180076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a57b0-9aeb-4371-a0cf-f61d0c872d99 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.228987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.229611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.229896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.230172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.230832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:25.231135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.232609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.245531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce8437-66b8-46e0-a537-8e5e3e4b8640 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.246459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 278/563] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:25.247330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.303472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.304428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.304717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.304996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.305763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:25.306074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.307564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.329245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 286/564] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:25.330034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.372684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.375638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4040da-c8e4-4a41-b349-e2d45348d88c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.408708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.418266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.418897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.419172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.419444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.430963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:25.436305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:25.437874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 279/565] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:25.445484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.460520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b597fa74-51fe-40ce-a78b-41a7ec36e0f8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.503961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.504772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.505134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.505481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.513267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.514835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:25.530701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:25.532085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:25.533221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 287/566] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:25.533827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.561935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98d872-7803-49bf-91e8-e4b14f3e1759 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.598254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.598888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.599173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.599452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.610758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:25.616275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:25.617737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 280/567] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:25.619021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.644107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.664232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56dbec-c4be-4b40-9e66-578ee710a8ba None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.691598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.703560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.704243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.704517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.704789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.714998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.716744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:25.733589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:25.736646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:25.738412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:25.738412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:25.738412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:25.738412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 288/568] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:25.738850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb2614-73c8-49fd-8885-20dabbcdbc60 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.738985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.776132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.776746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.777022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.777291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.777962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:25.778260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.779657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf8287a-64e8-4a4d-bc64-a38323eac184 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.789047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 281/569] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:25.790256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e3a3-4743-4cbf-879c-7946992477ef None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.790539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e3a3-4743-4cbf-879c-7946992477ef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.790814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89e3a3-4743-4cbf-879c-7946992477ef None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.792027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 282/570] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:25.793244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e3a3-4743-4cbf-879c-7946992477ef None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.853226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d0436-b97e-4153-8a58-982cf4657bdf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.881216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.891102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.891696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.892030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.892274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.892960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:25.893266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.894684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.900736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89e3a3-4743-4cbf-879c-7946992477ef None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:25.906650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 289/571] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:25.907463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:25.934410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:25.935011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:25.935286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:25.935555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:25.936244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:25.936543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:25.938484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:25.946792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:25.946934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 283/572] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:25.947631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.016192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa20ea-9632-498c-94fe-7855d1840b79 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.043464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722470d8-2505-40ab-89e9-f27fc3af305e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.050548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.051155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.051437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.051711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.052379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:26.052702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.054053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8534da5f-e94b-42be-b3b9-ceee3badd189 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.065521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 290/573] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:26.066724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.067005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.067513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.078546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.079150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.079437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.079708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.084695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:26.085879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:26.089727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.091425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.094378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:26.108018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:26.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:26.110836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:26.110836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:26.110836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:26.110836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 284/574] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:26.110836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.153370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 291/575] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:26.154206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.180637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.214170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc869eb3-2d19-4d78-84cc-0049d8279df2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.250504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.251096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.251373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.251641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.252374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=10831ea9480d488ab2a81bece14ff090) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:26.252680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.253138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8558237-79af-419a-b9f8-93f1161d2f1b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.254109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.265720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 285/576] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/role_assignments?user.id=10831ea9480d488ab2a81bece14ff090&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:26.265720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.299511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.300441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.300901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.301426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.314273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.316301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:26.334490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:26.335567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:26.336496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:26.336496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:26.336496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:26.338224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.345278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 292/577] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:26.346511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.386639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c4e70c-fb3b-412a-9c5f-5a81124fcc57 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.428112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.428753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.429032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.429329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.436284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.437902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.440289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:26.450675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:26.451767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:26.453305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 286/578] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:26.453794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.469728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b46c7-6e38-4733-a98a-085ca537bf09 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.505632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.506584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.507036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.507451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.508413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:26.508944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.510339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.523834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 293/579] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:26.524687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.576876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ea9d9-22fc-4772-af74-cf80223f3d22 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.599449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.616569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.617304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.617640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.617981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.618692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:26.619057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.620541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.635948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae84552-5ce4-48f0-9b5c-f84e7c2f8b6c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.642439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 287/580] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:26.642439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.686678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.687787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.688204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.688581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.698357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.703608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:26.708801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:26.709783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:26.710881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:26.712885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 294/581] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:26.713831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.747407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f69b7c-e86e-4c67-8e4e-a2322db05e15 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.791621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.792344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.792685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.793025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.793816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:26.794187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.795685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.813474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 288/582] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:26.813474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.834613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47c19d-a2d3-4f87-a3b3-2544185a1783 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.857440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.871689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.872346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.872629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.872951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.880968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.882391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:26.894621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:26.895747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:52:26.896916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:26.900291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 295/583] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:26.901099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:26.926647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f90d1-c4c6-4ecf-8d53-40287a8f4849 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:26.961711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:26.962576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:26.963004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:26.963176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:26.963300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:26.972592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:26.973951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:26.985261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:26.986223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:26.987178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 289/584] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:26.987780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.015838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7d070-2ed1-443e-8af7-32ee87924f06 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.049666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.050262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.050542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.051118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.051777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:27.052113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.053459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.061429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 296/585] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:27.062863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.091951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:27.095624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ae5c1-ce60-4789-a348-2ea03df7b9f7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.138811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.139629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.141347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.141632 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/508f557ac5e34c67a31e4d55f8142a0f/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.161300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=508f557ac5e34c67a31e4d55f8142a0f, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.163410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.170466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a058c5dd-9587-4667-8637-ac7f0c5f11fb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.178528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 290/586] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:26 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/508f557ac5e34c67a31e4d55f8142a0f/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:27.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.193436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:27.210879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.211601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.211887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.212278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.213090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:27.213404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.214936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.237452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 297/587] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:27.238755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.368089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e12ca-269e-4c94-a1a8-b5b023cf73e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.403759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbe866-070f-46b7-b5d8-8c225caf499f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.410686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.411166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.411277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.411523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.412259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:27.412563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.414084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.434963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 291/588] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:27.440440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.453468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:27.459403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.460132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.460602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.460602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.476089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.476089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:27.489794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:27.490983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:27.492171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:27.492171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:27.492171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:27.492171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 298/589] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:27.492171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.524706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:27.567684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11790c7-95d8-45c9-919d-60b728657874 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.599765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26039216-999f-4c53-8980-f706de926bc5 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.613340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.614207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.614582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.615019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.631502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.635971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.638319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.639017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.639291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.639631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.653169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.654901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:27.661623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:27.662669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:27.663746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:27.663746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:27.663746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:27.665745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 292/590] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:27.666963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:27.679003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:27.680103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:27.681165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:27.681165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:27.681165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:27.681165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 299/591] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:27.681402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.756495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:27.791333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:27.822131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67230bfb-0fd4-4da8-a9cc-b343328bc74a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.824156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade43f4-e04a-47fb-9662-e663aff4c581 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:27.883936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.884812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.887720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.888200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.889098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:27.891028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:27.891969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:27.892249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.892454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:27.892868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:27.893926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:27.894346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:27.895835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.897366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:27.919003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 293/592] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:27.920929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:27.926459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 300/593] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:27.927514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.036551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192db853-c41e-44e6-9c8c-a1ef9818fb88 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.037446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd1572-5485-4872-ac70-2315645419e0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.058859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.071690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.072338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.072616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.072886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.076892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.077506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.078387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.078668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.079337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:28.079624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.080692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.081020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.082117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:28.095491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:28.096472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 301/594] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:28.097929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.103159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 294/595] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:28.103159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.103521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.205357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5b92c-a641-4cf4-8b79-ac4196becd13 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.221464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351cfe-33bb-49c6-ab35-022f9b41e6a0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.240718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.241338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.241613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.241886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.248639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.250106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.259017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.259654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.259936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.260237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:28.261434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:28.263366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:28.266107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:28.271254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 302/596] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:28.271482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.271652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.271813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:28.283439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:28.285662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:28.288082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 295/597] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:28.295897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.333191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.338212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.375058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c834ac-d5f7-48ec-a960-0d2a267cdd83 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.412561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c777d1-c69d-460e-81fe-0d23eccdac4f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.417607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.418091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.418448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.419006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.419538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:28.419905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.421529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.444205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 303/598] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:28.444205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.460939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.461613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.461940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.462333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.463193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:28.463532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.465514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e94954f-a98d-41ab-bf90-bf4863ae3df5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.480061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 296/599] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:28.481553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4309f2-e253-4ad7-9cd1-63289b3416e7 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.481919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4309f2-e253-4ad7-9cd1-63289b3416e7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.482300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4309f2-e253-4ad7-9cd1-63289b3416e7 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.488053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 297/600] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:28.488053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4309f2-e253-4ad7-9cd1-63289b3416e7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.586527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4e1c0-0f7c-4aa4-b868-1365d98270ef None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.616161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.625834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.632212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4309f2-e253-4ad7-9cd1-63289b3416e7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.639135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.639899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.640228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.640491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.641322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:28.641760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.643616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.658862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 304/601] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:28.659859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.684766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.685842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.686833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.687365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.700785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.703760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b368d6dc-5e1b-4148-8a23-0789dd32faa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:28.724027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:28.725194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:28.726254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:28.726254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:28.726254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:28.730185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 298/602] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:28.731544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.731978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.732228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.752814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:28.754013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:28.791391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8f07b-6400-480c-9d23-dad1aeacf3d7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:28.827157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 299/603] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:28.828422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.845973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:28.847093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:28.848050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:28.848546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/72251fea0df54206b554236c14296a7b/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:28.869478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=72251fea0df54206b554236c14296a7b, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:28.872270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:28.894641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 305/604] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:28 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/72251fea0df54206b554236c14296a7b/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:28.895804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:28.935389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.941771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:28.986277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c53fdc8-2ff6-4086-978d-9db7a88a1abe None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.030438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b3060-039a-4b6e-8c81-f51cbc208dda None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.073783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.074699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.075189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.075644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.076502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:29.076984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.078710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.079138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.079552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.079899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.080279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/e1aaf56b3e654c0bb65d0c804fd797e6/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.093965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 306/605] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:29.101105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.126365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=e1aaf56b3e654c0bb65d0c804fd797e6, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.128193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.149839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 300/606] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:28 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/e1aaf56b3e654c0bb65d0c804fd797e6/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:29.152150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.245159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:29.256955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a71b-4bc3-43e3-a7cc-e10cbd89420f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.294751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f45a0d-d8ed-485d-8bc8-809c7304b12f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.296890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.297988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.298344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.298702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.307113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.308755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4cde32aa-3c98-4f27-b278-1346b019e802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:29.321946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:29.322953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:29.323857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 307/607] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:29.329946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3851-61d1-407c-af45-3a8a8748b59f None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.330278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3851-61d1-407c-af45-3a8a8748b59f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.330603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca3851-61d1-407c-af45-3a8a8748b59f None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.332245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 308/608] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:29.332897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca3851-61d1-407c-af45-3a8a8748b59f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.337636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:29.348711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464aa227-fff8-4b32-bc01-c56d44452aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.350172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aa227-fff8-4b32-bc01-c56d44452aee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.350667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aa227-fff8-4b32-bc01-c56d44452aee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.351137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464aa227-fff8-4b32-bc01-c56d44452aee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.367365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464aa227-fff8-4b32-bc01-c56d44452aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-464aa227-fff8-4b32-bc01-c56d44452aee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:29.374757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:29.375949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:29.376919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:29.378868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 301/609] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:29.380306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.380746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.381189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.401993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:29.402686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:29.471621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca3851-61d1-407c-af45-3a8a8748b59f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.492165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 302/610] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:29.493325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.533309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.533884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.534167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.534444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.535217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:29.535535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.537083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.570652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 309/611] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:29.589096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.623980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cfc1f8-7911-4964-b39c-3f68ecbd6797 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.629858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:29.656561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:29.667997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.668641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.669461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.669747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.670487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:29.670806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.672764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.683806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 303/612] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:29.689417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.849902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791f918-20d1-4c29-91d6-d277511229f4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.869308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eba228-3054-4d84-a27c-2042310f0416 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:29.899958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.900873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.901258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.901630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/49f7536740a54bdbba5b6c81f732c439/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.916888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:29.918242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:29.918674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:29.919354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:29.921012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=49f7536740a54bdbba5b6c81f732c439, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.923176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.932333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:29.934121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:29.948782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 310/613] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:29 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/49f7536740a54bdbba5b6c81f732c439/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:29.953370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:29.969780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:29.971035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:29.972174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:29.972174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:29.972174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:29.973838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 304/614] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:29.982451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.061643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:30.093928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:30.146080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b50e32-c0bf-4b9d-afd0-a5a0cca8fec6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:30.183154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6f5e3-7e94-47a2-a950-dafd1b94f03e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:30.198657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:30.199307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:30.199586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:30.199857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:30.218095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:30.224862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:30.225986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:30.226996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 311/615] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:30.231664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.232382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:30.233158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:30.233577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:30.234582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:30.235516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:30.236315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:30.238916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:30.255445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 305/616] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:30.261685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.412294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:30.421715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f04d5a-7033-4a06-b4e0-b2f5f51dec5a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:30.429400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fceb8d-240b-4387-a42b-a27e99afb5f7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:30.471155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:30.493170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:30.494564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:30.495012 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:30.495428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:30.497762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:30.500345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:30.501533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:30.501969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:30.506635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:30.507106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:30.508722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:30.516087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:30.517598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:30.519241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 312/617] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:30.525509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:30.539714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:30.540960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:30.542089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:30.545623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 306/618] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:30.551727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.726219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e58f4-7430-495f-9fe7-1aeb0985ff8a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:30.743004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e2ba9-2810-45c1-acc7-02327cb79c70 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:30.790861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:30.791492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:30.791767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:30.792059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:30.803372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:30.803984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:30.804283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:30.804556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:30.805288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:30.805596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:30.807043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:30.818233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:30.819980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:30.843556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:30.844985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:30.845892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:30.845892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:30.845892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:30.846675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 307/619] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 10:52:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:30.846774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 313/620] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:30.853066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.854159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:30.873160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:30.946566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:30.982026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb6ac-9a70-49b0-89e9-49bd532cfdaa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.024641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.025525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.025957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.026411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.041632 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.043942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.064304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de587d-7ce3-454b-b794-08170013e2c1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:31.065591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:31.066682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:31.067774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:31.067774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:31.067774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:31.069893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 308/621] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:31.076066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.149536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.150477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.151806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.152352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.159195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:31.159700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.161349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.194704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 314/622] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:31.195891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.230620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:31.269352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9390-16b2-4e3d-9132-12b4e780ddd0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.318061 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:31.328891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df15bc5-cf1b-4dd8-8ca7-c77c2c0c86ec None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.347779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.348425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.348711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.348986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.349636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:31.349945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.351322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b60e7-e1cf-4940-b43c-e8eebaccf338 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.363551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 309/623] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:31.366059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.366695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.366977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.367302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.369086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026966fb-7380-4397-b7bd-2c713b616c67 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.369376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026966fb-7380-4397-b7bd-2c713b616c67 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.369652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026966fb-7380-4397-b7bd-2c713b616c67 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.371019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 310/624] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:31.371899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30a1a6-e3ed-4faa-9c30-95038af0a9a8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.372263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30a1a6-e3ed-4faa-9c30-95038af0a9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.372536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30a1a6-e3ed-4faa-9c30-95038af0a9a8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.373797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 311/625] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:31.375454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30a1a6-e3ed-4faa-9c30-95038af0a9a8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.379894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.381415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-6b29e354-d1a7-4733-8281-2c8916e3e2a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:31.393884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:31.394860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:31.395790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 315/626] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:31.396410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e757866-b01e-48f5-adf9-0bb6daff3f00 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.398075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e757866-b01e-48f5-adf9-0bb6daff3f00 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.398075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e757866-b01e-48f5-adf9-0bb6daff3f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.415679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2e757866-b01e-48f5-adf9-0bb6daff3f00 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:31.417027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2e757866-b01e-48f5-adf9-0bb6daff3f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:31.493009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 316/627] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:31.494318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.494609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.494881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.505846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30a1a6-e3ed-4faa-9c30-95038af0a9a8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.521091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:31.522355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:31.536673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:31.548384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.549532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.550028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.550853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.560085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.563806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:31.579177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:31.580146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:31.581128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:31.587197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 312/628] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:31.587197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.608052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 317/629] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:31.608052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.635707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:31.790248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e797bb-ac0d-43ff-ad69-eba35dd2b8bc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.801135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1db361-181b-427e-a1c3-f3af0d3f35ba None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.829771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.830205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.830493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.830825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.831563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:31.831896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.833446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.838979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:31.839878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:31.840666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:31.841138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:31.842030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:31.842533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:31.844445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:31.854260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 313/630] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:31.855249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.857527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 318/631] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:31.864209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:31.906368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:31.945615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:31.966005 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3652f6-ac3e-4562-a93b-bdb17ba91a32 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:31.973028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd58ea-859e-4b47-9665-c29f35ee7778 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.007260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.007896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.008209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.008483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.010110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.010728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.011008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.011282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.019912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:32.025208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:32.026131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:32.027052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 314/632] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:32.028412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.086241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.087590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.131302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad38c35-a7d8-4719-bf23-eb738bf5be27 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.133687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.163985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.170224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.170882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.171213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.171551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/10831ea9480d488ab2a81bece14ff090/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.185721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=10831ea9480d488ab2a81bece14ff090, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.187723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.190691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 319/633] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:32.191511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.202974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 315/634] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:32 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/10831ea9480d488ab2a81bece14ff090/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:32.204404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.301645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd86b70-cd7c-4af0-b6ef-4ffded1b893f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.335332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0d085-cb2f-4f4a-9834-e3f1be977f94 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.344359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b3c6c-9b44-4491-b16c-431db72279cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.344979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b3c6c-9b44-4491-b16c-431db72279cc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.345202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b3c6c-9b44-4491-b16c-431db72279cc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.345474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b3c6c-9b44-4491-b16c-431db72279cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.360838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b3c6c-9b44-4491-b16c-431db72279cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9b4b3c6c-9b44-4491-b16c-431db72279cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:32.365658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:32.366592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:32.367825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 320/635] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:32.368814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed75c67-ab50-466d-8a67-9d099becb176 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.368814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed75c67-ab50-466d-8a67-9d099becb176 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.368814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed75c67-ab50-466d-8a67-9d099becb176 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.370028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 321/636] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:32.370740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed75c67-ab50-466d-8a67-9d099becb176 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.379167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.380187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.380676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.381171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.382080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:32.382461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.383909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.392310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 316/637] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:32.393042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.412335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.412709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.487146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed75c67-ab50-466d-8a67-9d099becb176 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.501247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a56f2-282b-4358-a25b-b9276201638b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.527303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.527938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.528246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.528575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.537173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.537774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.538047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.538315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.538977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:32.539277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.540657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e133fd-b389-44f0-8d89-109c176258af None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.540928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.542571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.549433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 317/638] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:32.550643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.550925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.551198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:32.563151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:32.564241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:32.565401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:32.565401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:32.565401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:32.566791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 322/639] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:32.567693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.569720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:32.570903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:32.650596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 318/640] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:32.651371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.668781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.682054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.703346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9be5a5-eab1-4c2c-b03e-53d8969b9970 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.744085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.744796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.745082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.745357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.760525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.762240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.767006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b006d6-0d22-4901-a432-e547e2a1d42f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:32.779697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:32.781578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:32.782477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:32.782477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:32.782477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:32.786385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 323/641] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:32.787337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.822770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.823468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.823752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.824044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.840975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.842626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:32.859943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:32.861027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:32.861930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:32.861930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:32.861930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:32.861930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 319/642] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:32.862185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:32.914684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c265b-06c7-4855-a2f6-cfe77556b1e9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.928837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.956287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:32.957933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:32.960423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:32.960857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:32.961943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:32.962455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:32.965565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:32.981634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:32.989297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 324/643] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:32.990133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f539e686-5376-4688-8f13-a87cd5aa8cd7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:32.990323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.029595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.030212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.030591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.031025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.031884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.032353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.033768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.050420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 320/644] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:33.050420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.104751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29852eb5-0e08-4ab0-a408-142c0e127d29 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.141541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.142301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.142677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.143049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.143816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.144242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.145701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.160984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 325/645] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:33.162683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.176569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273371d-93b8-4f15-8709-bc76b3287c22 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.206937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:33.223991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.224673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.225283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.225567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.232728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.234681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.242551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:33.246590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:33.247504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:33.248683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 321/646] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:33.249769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.294576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980d222-1843-4efe-9f25-e6e35fdc560b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.336186 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.336818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.337101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.337377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.348014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:33.353224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:33.354147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:33.354986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 326/647] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:33.356730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.373429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992d2730-38cd-4c88-b7d1-b4e200b43fbd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.412555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.413182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.413462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.413737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.424231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.425762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:33.439619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:33.440746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:52:33.441767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:33.445374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 322/648] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:33.446208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.471650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421eb52-8034-40b5-b8bc-e1b7caa7c276 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.491184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:33.494621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:33.514355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.515336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.515791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.516397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.525253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.526700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:33.538440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:33.540377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 10:52:33.542160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:33.544671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 327/649] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:33.545458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.582452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deba456-bb08-4460-8521-29115c8e5bd0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.623152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.623939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.624339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.624625 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.625299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.625606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.627081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.639298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 323/650] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:33.640466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.674401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97092be8-dd0c-487f-844e-2a3826338a03 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.708717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.709237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.709515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.709803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.710446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.710748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.712136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.722018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 328/651] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:33.722823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.748964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:33.752222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e00d6-6c99-4b97-9b94-964ff564213a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.775355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:33.804719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.805485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.805766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.806812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.807548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.807859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.809323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.828166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 324/652] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:33.829013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.844024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4677a39-0f95-4b33-a102-f719c186b680 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.880630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.881648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.882177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.882743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.883760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.884313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.886102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:33.915134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 329/653] 10.222.0.20 () {56 vars in 1137 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:33.916815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:33.954018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a8fd3-23cb-4f42-adf3-46d3965d0bcc None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:33.992671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:33.993186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:33.993461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:33.993729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:33.994451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=508f557ac5e34c67a31e4d55f8142a0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:33.994757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:33.996225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.009950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 325/654] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/role_assignments?user.id=508f557ac5e34c67a31e4d55f8142a0f&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:34.009950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.038937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.043628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.045486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cda4af-fe22-4dcb-b689-cec66de3531e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.103413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.104392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.104835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.105264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.120925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.122788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.137711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09975ed-0584-43a4-bf5e-b25a952c25d3 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:34.145833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:34.146974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:34.147998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:34.147998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:34.147998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:34.150482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 330/655] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:34.150896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.187660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.188313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.188676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.188862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.281385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952b691-ae3d-4f2b-89b9-c4eb1f6e74af None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.283576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.286179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.314743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.339442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696529d-2ba8-4e8f-93cf-486a11c36228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.340747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696529d-2ba8-4e8f-93cf-486a11c36228 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.341244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696529d-2ba8-4e8f-93cf-486a11c36228 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.341728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696529d-2ba8-4e8f-93cf-486a11c36228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.372819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.400450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 326/656] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:34.401687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.461000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696529d-2ba8-4e8f-93cf-486a11c36228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.462456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c696529d-2ba8-4e8f-93cf-486a11c36228 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.525245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f224c6-fe06-4476-be28-b517ebde5a26 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.551940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.566698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.567421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.567707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.567993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.568698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:34.569007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.570419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.578158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 331/657] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:34.579412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7a80-66d2-4344-be51-f4c4f8533b40 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.579698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7a80-66d2-4344-be51-f4c4f8533b40 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.579971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f7a80-66d2-4344-be51-f4c4f8533b40 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.584147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 332/658] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:34.584147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7a80-66d2-4344-be51-f4c4f8533b40 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.589606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 327/659] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:34.590745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.642552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.710846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f7a80-66d2-4344-be51-f4c4f8533b40 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.723070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f86c8-972b-4b40-a9da-0d3c157fcfad None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.752584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.753218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.753501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.753780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.758838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.759673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.760177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.760640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.765425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.767297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.777135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.778967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-b69fab26-a00d-4d3d-a15a-59a2944bf5ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:34.784357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:34.785343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:34.786265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:34.786265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:34.786265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:34.790255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 333/660] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:34.791681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.791971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.792278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:34.795720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:34.802568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:34.803488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:34.803488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:34.803488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:34.803488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 328/661] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:34.803488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.816193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:34.817403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:34.839468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.895383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 334/662] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:34.896590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:34.935189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44a766-06a9-466b-b584-995d81f493e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:34.940583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:34.979014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:34.980085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:34.980371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:34.980650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:34.990983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:34.993695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:35.009110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:35.010134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:35.011170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 329/663] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:35.011793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.030969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790314b-052d-48ec-bd3d-8cbb922483c9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.070633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.071600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.072151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.072634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.073613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:35.074176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.075744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.096192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 335/664] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:35.097481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.141467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:35.150810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c7470-f737-4487-92d8-b59b0473ace2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.196767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.197393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.197674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.197951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.198607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:35.198943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.200506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.215668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:35.216985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 330/665] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:35.218151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.229866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f660a5-7eb1-4085-ab35-4d63687cddc3 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.274114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.275311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.275943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.276609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.277807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:35.278419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.285799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.315139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 336/666] 10.222.0.20 () {56 vars in 1139 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:35.315139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.342864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c091e-c34b-41e4-9b8b-f9f111f9cc20 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.384876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.385511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.385789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.386073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.394525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.395992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:35.408779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:35.410453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:35.412178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:35.414646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 331/667] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:35.415503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.435944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:35.437490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c10b9-c0e0-46cc-80d8-d766efa151a7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.481852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.482771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.483200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.484183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.495632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:35.501524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:35.502640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:35.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:35.504768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 337/668] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:35.505397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.518107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:35.550366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0728d75-a464-4cf8-a778-a92e557c34a1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.588331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.588960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.589660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.590479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.598737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.600233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:35.612095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:35.613214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:35.614157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 332/669] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:35.614659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.621506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d950a2b-c5ee-45fe-b996-211f9518045e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.672759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.673388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.673665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.673942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.706780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:35.741149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c613446-a110-4a5b-8aaa-953ed625d8a5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:35.757817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.759368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.772417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:35.796163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:35.796854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:35.797200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:35.797538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:35.798309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:35.798686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:35.800234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:35.817403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 333/670] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:35.818390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.907675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 338/671] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:35.908571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:35.957337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce81571-4c32-4a08-8c3d-ef7456463d43 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.003112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.004652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.004944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.005222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.005968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:36.006279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.007687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.010075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 334/672] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:36.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.042044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91003ace-afae-4fd2-a3e2-3e10fe10a977 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.059721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.098682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.099340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.099551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.099813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.100632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:36.100904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.102385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.125004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 339/673] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:36.125004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.164983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d53bd2-eef4-4278-bbed-f28b7eaab4e9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.201444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.202055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.202944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.203193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.203927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=e1aaf56b3e654c0bb65d0c804fd797e6) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:36.204263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.205720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.211749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 335/674] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/role_assignments?user.id=e1aaf56b3e654c0bb65d0c804fd797e6&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:36.212620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.249668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932bbb-ee3d-4a12-a3f8-68057a01e45b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.296936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.297597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.297883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.298158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.305737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.310505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.312868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:36.334190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:36.335220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:36.336195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:36.336195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:36.336195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:36.336365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 340/675] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:36.337131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.369678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c798d544-9f1e-47c1-8f38-cbfc74e3a286 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.370632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.417379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.417942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.418223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.418487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.433546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.435022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2e66001e-343a-458d-bfcd-e44204bb5f8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:36.451224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:36.452478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:36.453737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:36.453737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:36.453737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:36.453737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 336/676] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:36.454025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4dd9-862b-4ff1-ab50-d632b6cfabe8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.455806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4dd9-862b-4ff1-ab50-d632b6cfabe8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.455806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4dd9-862b-4ff1-ab50-d632b6cfabe8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.455806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 337/677] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:36.456556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4dd9-862b-4ff1-ab50-d632b6cfabe8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.476487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d6817-7f42-4a4b-ad2b-1ff27b6175d1 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.519547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.520313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.520597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.520863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.521710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=72251fea0df54206b554236c14296a7b) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:36.522021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.523468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.540043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 341/678] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/role_assignments?user.id=72251fea0df54206b554236c14296a7b&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:36.540043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.624108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4dd9-862b-4ff1-ab50-d632b6cfabe8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.644175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.677906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.678518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.678918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.679258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.679915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.680938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:36.681403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.682990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09ce14d-4f8f-489c-90dc-669bba856b3d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.700081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 338/679] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:36.700382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f35b53-5e2c-486d-bce3-562c7d87e65a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.701692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d503894-b935-422b-8512-3c40525cd313 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.701692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d503894-b935-422b-8512-3c40525cd313 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.701692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d503894-b935-422b-8512-3c40525cd313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.731486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1d503894-b935-422b-8512-3c40525cd313 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:36.733053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1d503894-b935-422b-8512-3c40525cd313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:36.752285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.752909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.753190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.753468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.754123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:36.754431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.755818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.770851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 342/680] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:36.771862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.840504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 339/681] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:36.841313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d503894-b935-422b-8512-3c40525cd313 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:36.906302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80803ab-9594-4196-a963-7403001d3462 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.959105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:36.959105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:36.959384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d503894-b935-422b-8512-3c40525cd313 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:36.959464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:36.959464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:36.960353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:36.970030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:36.971612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:36.984400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:36.985390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:36.986305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 343/682] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:36.993215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.004373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.005024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.005303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.005576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.008628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:37.015507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:37.017017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:37.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:37.032610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:37.034794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 340/683] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:37.038230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.161888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b181033-a1ae-4dcf-b712-d0147e07b196 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:37.200689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.201364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.201690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.201963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.202766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:37.203084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:37.204754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:37.226902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 341/684] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:37.232626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.260665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:37.266492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b32a0d-9370-4d0a-8c34-ccd9e81106cb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:37.326885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.327549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.327831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.328145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.329387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:37.329858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:37.331625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:37.341000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53020c1-3074-49d5-a49b-14fcdd3a5176 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:37.359012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 344/685] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:37.363480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.394260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.394949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.395225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.395548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.454711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:37.478977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:37.480385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:37.512564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:37.522461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dae7f-9c8d-4348-a2e9-70b71c823443 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:37.590220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.591365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.591827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.593439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.613812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:37.625220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:37.626453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:37.627482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:37.628367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 345/686] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:37.632036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.668773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 342/687] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 10:52:37.670209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.803070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0d6105-f708-44be-b0dc-4cff9db78d38 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:37.816801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc62c98-9b98-44e3-b0ff-3c60e2099d4e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:37.833418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:37.839715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:37.847062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.848055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.848506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.848927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.849837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=49f7536740a54bdbba5b6c81f732c439) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:37.850296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:37.851936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:37.857514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:37.857847 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:37.858194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:37.858691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:37.859416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:37.859720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:37.861212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:37.868792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 346/688] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/role_assignments?user.id=49f7536740a54bdbba5b6c81f732c439&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:37.870082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:37.871994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 343/689] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:37.877932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.001343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7b3b-4ae9-4b0d-8bff-471413c0ef08 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.050884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.051448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.051723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.051991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.070829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.072359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:38.101585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:38.102784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:38.103788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:38.103788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:38.103788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:38.103788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 344/690] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:38.106438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef929fee-2485-4248-8c91-98da31095630 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.107884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.150893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:38.156913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.158745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.159320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.160582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.176660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.178953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:38.199818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:38.201021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:38.202913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:38.202913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:38.202913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:38.205661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 347/691] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:38.211382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.235863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f1b4d-2b47-438e-8b61-8e89007ba545 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.274579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:38.287619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.288425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.288709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.289100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.289865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:38.290178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.291573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.305820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 345/692] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:38.317221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.338869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121605e5-4b1a-4861-a0d7-c7c28eb108f8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.393831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.393831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.393831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.393831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.394334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:38.394334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.394334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c7938-9fae-4119-bcf8-fc524fc8930c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.412030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 348/693] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:38.412030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442279-7e06-4332-a4d0-7e9a4dc34799 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.412030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442279-7e06-4332-a4d0-7e9a4dc34799 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.412030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43442279-7e06-4332-a4d0-7e9a4dc34799 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.412030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 349/694] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:38.418839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.419136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.419402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.442134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:38.443865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:38.458356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:38.467614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dd221-afe2-45f5-a777-e16f2879333b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.505667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.506411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.506698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.507022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.516221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.517811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:38.530905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:38.531922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 346/695] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:38.533294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.537284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 350/696] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:38.545098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.621178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:38.751912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a656ef29-9c26-4574-b250-ae3d1814677d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.766876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da85d7-27e8-4329-8da9-62c1dbde48cd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:38.800309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.800943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.801225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.801503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.802263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:38.802574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.804054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.837828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 351/697] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:38.842394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.860378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:38.863303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:38.863303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:38.863303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:38.873214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:38.874832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:38.888387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:38.889361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 347/698] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:38.890710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:38.913756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:38.958463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:38.972212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2d93b-a067-4d52-a3ee-11d04e71d2e0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.011233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.011847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.012161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.012436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.020928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83db4f78-0888-460f-bb9a-9063a0a368dd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.060458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.061550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.062030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.062527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.063129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.064543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cdbb9308-ab65-48c3-80ca-3eb125b9f422 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:39.075911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:39.077207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:39.078430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:39.081976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 348/699] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:39.083209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ab7a0-6f43-4546-bb32-02f27916143f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.083497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ab7a0-6f43-4546-bb32-02f27916143f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.083777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ab7a0-6f43-4546-bb32-02f27916143f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.085102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 349/700] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:39.085804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ab7a0-6f43-4546-bb32-02f27916143f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.094598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.096180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.173783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:39.198058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:39.198833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ab7a0-6f43-4546-bb32-02f27916143f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.217356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 352/701] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:39.217356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.241865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.242568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.242836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.243116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.243813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:39.244150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.245598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65024699-fa1e-4c5a-8045-e1e269135e39 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.258495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 350/702] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:39.262562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.262954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.263302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.284361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:39.285806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:39.328752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491e6c7f-7a2a-401b-8b33-b3d8b9ddeecc None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.366956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 351/703] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:39.367187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.367835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.368166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.368444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.369066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.369185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:39.369377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.370744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.383548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 353/704] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:39.384493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.454725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:39.492429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:39.502868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380caa88-1861-4f81-8485-84ad87fa5eee None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.503982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695f169-fccd-4111-b6a2-a8f1a736c059 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.539312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.539680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.539950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.540274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.540552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.540728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.541106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.541479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.542307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=10831ea9480d488ab2a81bece14ff090) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:39.542863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.544503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.551716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 354/705] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/role_assignments?user.id=10831ea9480d488ab2a81bece14ff090&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:39.552124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.552847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.553828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:39.572249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:39.573478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:39.574639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:39.574639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:39.574639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:39.574639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 352/706] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:39.580370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.677524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2ea8d-6382-46b9-bde2-e88e62759643 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.700186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626d3143-41c9-4210-be6c-82762b9606b8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.703912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:39.729951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:39.735952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.737697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.738301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.738783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.739924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:39.740970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.743183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.750192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.750944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.751227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.751641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 355/707] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:39.752228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.752560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.753355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:39.753882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.755505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.775241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 353/708] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:39.789098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.890836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588116a2-e481-4394-ba15-6b92c6dfccf6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.917764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa54585a-3ada-4ade-b5eb-0ce1e17c21f0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:39.930283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.931218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.931696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.932175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.945388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.947195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.958887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:39.959623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:39.959967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:39.960418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:39.965768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:39.966807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:39.967718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:39.967718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:39.967718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:39.969724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:39.970292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 356/709] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:39.973319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:39.976667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:39.989890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:39.990807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:39.996652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 354/710] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:40.000096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:40.040382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:40.046500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:40.103057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db35c8-6033-4787-ac59-f65266c0c458 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:40.114732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f1623-a232-493a-8ea0-96011dc80ddb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:40.145231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:40.146252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:40.146813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:40.147290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:40.148177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:40.148667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:40.150254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:40.152758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:40.152758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:40.153051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:40.153129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:40.153986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:40.154910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:40.155989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:40.169437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 357/711] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:40.169987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 355/712] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:40.178008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:40.179332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:40.284752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:40.287983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b45e47-2ae5-45ce-ba6c-7191888286a5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:40.288437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:40.290628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbdc5e2-d9bd-4918-ab04-e91243cfc8c2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:40.326746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:40.327422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:40.327680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:40.327950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:40.328840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=508f557ac5e34c67a31e4d55f8142a0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:40.329153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:40.330579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8ced9-3f12-4f82-8c40-80a198aa0ba7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:40.332143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e1785-0da3-4ea3-bb7a-83cecacda5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:40.333412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e1785-0da3-4ea3-bb7a-83cecacda5b4 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:40.334007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e1785-0da3-4ea3-bb7a-83cecacda5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:40.334476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e1785-0da3-4ea3-bb7a-83cecacda5b4 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:40.335336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e1785-0da3-4ea3-bb7a-83cecacda5b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:40.336982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e1785-0da3-4ea3-bb7a-83cecacda5b4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:40.344255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 356/713] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:40 2022] GET /identity/v3/role_assignments?user.id=508f557ac5e34c67a31e4d55f8142a0f&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:40.379051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 358/714] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:52:40 2022] POST /identity/v3/users => generated 254 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:40.438403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee157355-f204-4b5f-a6f4-a085b00a32dc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:40.438683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee157355-f204-4b5f-a6f4-a085b00a32dc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:40.439029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee157355-f204-4b5f-a6f4-a085b00a32dc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:40.440516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 357/715] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:40.446605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:40.447202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:40.447654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:40.485480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:40.487062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:40.591876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 359/716] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:52:40.877090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee157355-f204-4b5f-a6f4-a085b00a32dc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:41.003120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee157355-f204-4b5f-a6f4-a085b00a32dc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:41.054122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:41.055047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:41.055481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:41.055898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] PATH_INFO: `/v3/services/network` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:41.056650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:41.058186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:41.073406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:41.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:41.075758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:41.083091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 358/717] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:40 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:41.084437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:41.237636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:41.250586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879c47d6-b261-44f3-8996-27f259c21ec9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:41.288850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:41.289482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:41.289764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:41.290039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:41.290696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] RBAC: Adding query filter params (name=network) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:41.290999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:41.292441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07079d5-e418-4ccc-8a15-8a09d7a9f03d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:41.297965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 360/718] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:52:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:41.305211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:41.342013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:41.435676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc20b5-d436-41ca-91aa-f473e44a0419 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:41.481787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:41.482399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:41.482679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:41.482950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:41.484297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:41.484595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:41.485972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b56a3-0ad9-497b-81f4-80be97e006d5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:41.494793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 359/719] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:52:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:42.120295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64c3f-c983-46da-85e8-a5e11b57705a None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.120794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64c3f-c983-46da-85e8-a5e11b57705a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.120794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be64c3f-c983-46da-85e8-a5e11b57705a None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.122398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 361/720] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:42.127920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabcdab-4616-4bbb-9399-93d59475edcb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.128401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabcdab-4616-4bbb-9399-93d59475edcb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.128586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fabcdab-4616-4bbb-9399-93d59475edcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.153172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9fabcdab-4616-4bbb-9399-93d59475edcb None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:42.154424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fabcdab-4616-4bbb-9399-93d59475edcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:42.254495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 360/721] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:42.303426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acda85f-0f07-4acc-9f03-93fadb07bcd4 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.303735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acda85f-0f07-4acc-9f03-93fadb07bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.303976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acda85f-0f07-4acc-9f03-93fadb07bcd4 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.305694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 362/722] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:42.310838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac651e90-5abb-4ad0-8cf5-0c1bde5c428a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.311157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac651e90-5abb-4ad0-8cf5-0c1bde5c428a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.311435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac651e90-5abb-4ad0-8cf5-0c1bde5c428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.334472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ac651e90-5abb-4ad0-8cf5-0c1bde5c428a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:42.335682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ac651e90-5abb-4ad0-8cf5-0c1bde5c428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:42.358550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:42.419597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 361/723] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:42.550589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acda85f-0f07-4acc-9f03-93fadb07bcd4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:42.647414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a5615-46b4-436d-a34a-7f83827f64ae None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.647799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a5615-46b4-436d-a34a-7f83827f64ae None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.647976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a5615-46b4-436d-a34a-7f83827f64ae None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.649459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 362/724] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:42.656577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.657157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.657629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.679980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:42.681863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:42.718999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acda85f-0f07-4acc-9f03-93fadb07bcd4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:42.800778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:42.801938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:42.803517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:42.803790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:42.804503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:42.805886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:42.818113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 363/725] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 10:52:42.819283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:42.844215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 363/726] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:52:42 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:42.952921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:42.978601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c454e7b-6f00-4db3-8298-12b1a2e07911 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.047010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.047666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.047945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.048242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.068641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:43.073682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:43.074688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:43.075947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 364/727] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:43.082949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.092096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:43.120831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.239768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5822d14-eee1-4f5b-be82-dd7ca854d6ed None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.274174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc06fc-d141-4447-aa87-41c51f5d1f11 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.283500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.284231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.284527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.284790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.285955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:43.286269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:43.288120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598e3068-9159-47ea-b926-13b141233d01 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:43.297684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 364/728] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:43.299723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8922-a793-4806-8c8d-e16f45393dad None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.300091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8922-a793-4806-8c8d-e16f45393dad None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.300494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f8922-a793-4806-8c8d-e16f45393dad None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.302322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 365/729] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:43.307190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f8922-a793-4806-8c8d-e16f45393dad None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.318027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58670eb-5e8d-4ac3-957c-2ab2d7630fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.318677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58670eb-5e8d-4ac3-957c-2ab2d7630fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.319011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58670eb-5e8d-4ac3-957c-2ab2d7630fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.319289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58670eb-5e8d-4ac3-957c-2ab2d7630fa9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.329490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:43.334451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58670eb-5e8d-4ac3-957c-2ab2d7630fa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e58670eb-5e8d-4ac3-957c-2ab2d7630fa9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:43.341952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:43.343472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:43.344364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:43.345307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 365/730] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:43.346203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.346593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.346962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.371845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:43.374061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:43.384896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:43.423277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f8922-a793-4806-8c8d-e16f45393dad None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.459962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.460735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.461089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.461440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.461640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 366/731] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:43 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:43.462409 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.492386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:43.493868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:43.526702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:43.527724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:43.528672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:43.528672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:43.528672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:43.531749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 366/732] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:43.540107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.588754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f3cbf-583e-43a1-8bb5-88fe77a2a37d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.622609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:43.632156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.633203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.633698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.634192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.635168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:43.635775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:43.637501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:43.647963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 367/733] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:43.652975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.752264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc53db6-8a6e-4ff0-b4d7-2abb59f629aa None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.791199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:43.816219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.816732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4e3a4-8796-4847-971d-91893b9ed05c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:43.817429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.817943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.818605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.820368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:43.820894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:43.822501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:43.844608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 367/734] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:43.845442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.860075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:43.860740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:43.861019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:43.861294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:43.872297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:43.874135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:43.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:43.892405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:43.893320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:43.893320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:43.893320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:43.893320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 368/735] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:43.893559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:43.948525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:43.986296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c3afd-cd87-4f29-934b-2cff573f3b4c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.004177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9c840-37dc-401e-a2a4-c935646e1b77 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.034654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.036249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.036743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.037288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.040047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.040698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.041167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.041289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.049524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.050981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.051734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:44.058066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:44.059025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:44.059942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-42fe43ff-2858-40ec-809f-7b60a0ec7504 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:44.068507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:44.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:44.070411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:44.070938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 368/736] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:44.071011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 369/737] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:44.071079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.071148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761078-df28-4789-b612-3ddc8c9351d2 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.071148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761078-df28-4789-b612-3ddc8c9351d2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.071148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef761078-df28-4789-b612-3ddc8c9351d2 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.071148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 370/738] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:44.071148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761078-df28-4789-b612-3ddc8c9351d2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.116966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.182654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a7757-7909-4702-a4fd-d67175cb9850 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.195528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef761078-df28-4789-b612-3ddc8c9351d2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.197276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.223822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.224467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.224781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.225021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.225682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:44.226195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.227347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.236849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.237176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.237371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.237646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.238468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:44.238751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.240114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56879e7-5b56-40ab-9332-c0eab2159170 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.240744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 369/739] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:44.245840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.265706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 371/740] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:44.265706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.265706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.265706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.282131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:44.283332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:44.360063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 372/741] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 10:52:44.360063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.377399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.383353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66eb077-1e87-4ec3-aace-aa711cd70d49 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.425210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.425992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.426373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.426754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.427652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:44.428114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.429677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.446377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 370/742] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:44.446377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.503286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.518606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b77941-71d5-4935-90fb-1c95a43fce18 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.566067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.567287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.567937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.568401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.570085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e103ca8-92e8-498a-8b82-48d5138143bf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.578151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.580366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:44.598007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:44.599049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:44.599996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:44.604146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 373/743] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:44.605365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.618906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.619611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.619899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.620230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.661317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.709383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.710850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.734744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1f4b1-374c-430f-8a17-c7613f80442a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.781487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.782082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.782364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.782636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.795191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.797274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:44.812947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:44.813974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:44.814877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:44.814877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:44.814877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:44.814877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 374/744] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:44.814877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.821433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.839869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 371/745] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 10:52:44.841059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:44.916337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:44.940947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e87c12-a177-4c60-a9fb-92f174f44294 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.963818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25973ea9-045b-41ae-95c9-7dab2bd895ef None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:44.979470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:44.980116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:44.980397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:44.980668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:44.981408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:44.981713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:44.983976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.002192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.002806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.003087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.003361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.008883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 375/746] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:52:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:45.009139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.009226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.009300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.023062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 372/747] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:52:44 2022] POST /identity/v3/services => generated 240 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:45.023947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.172573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839e14c-a3d9-4672-803c-36174b674841 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.200201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:45.223041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.223956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.224262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.224535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.225958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:45.240575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.241750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:45.261717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:45.262912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:45.263901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:45.263901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:45.263901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:45.263901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 373/748] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:45.263901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.284568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbb5a8-d00d-42e1-a914-e23372ba479e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.321993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.322704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.322993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.323266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.323911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:45.324255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.325940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f942fc-e019-4e71-9505-db47f838cc96 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.338646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 376/749] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:45.341307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf5b42-0972-4d58-8cad-7198b3c84d7f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.341744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf5b42-0972-4d58-8cad-7198b3c84d7f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.342158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abf5b42-0972-4d58-8cad-7198b3c84d7f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.344672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 377/750] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:45.345607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf5b42-0972-4d58-8cad-7198b3c84d7f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.377287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf0859-77c8-41c4-95c0-9f46da2942fe None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.416699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.417328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.417670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.417950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.432233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.433618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:45.449398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:45.450427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:45.451324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:45.451324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:45.451324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:45.455527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 374/751] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:45.456420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.460640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abf5b42-0972-4d58-8cad-7198b3c84d7f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.480186 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:45.487680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:45.505504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.506118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.506398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.506666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.507310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:45.507614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.509041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1a1505-b427-4052-a23d-f5cb8dbe0fb0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.521693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 378/752] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:45.522919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.523205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.523483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.543835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:45.545070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:45.575569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e461495-3202-4cdb-a960-92abf7f2fda9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.616219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.616865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.617145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.617418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.624382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 379/753] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:45.625102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.634176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.636591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:45.653439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:45.654465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:52:45.655948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:45.659462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 375/754] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:45.661265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.744113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcd6ff-a32b-445e-bb3c-ede13f558ee4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.756153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:45.785091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.787120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.787552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.787976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.789257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:45.789994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.791654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:45.792519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.814836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 380/755] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:45.814836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.821506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7ad46-7580-44ad-ab55-068cb7847b5a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.862275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.862935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.863215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.864279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.875050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.877615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:45.895142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:45.896478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:45.897615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:45.900787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 376/756] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:45.902086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:45.946953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4dd94-7659-461a-b286-39e2e49605fa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:45.990335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:45.990914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:45.991201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:45.991475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:45.992227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:45.992536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:45.993968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.020396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 381/757] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:46.021825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.043653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a833c-69d2-4bf4-8dc0-355b6a92c86a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.053558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.085856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.086618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.087230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.087515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.097664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.099720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.102319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:46.113714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:46.114660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 377/758] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:46.115558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.137817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb1629-985e-4707-88be-659a75a4fb41 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.176176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.176902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.177179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.177452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.178174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:46.178481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.179964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.193300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 382/759] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:46.194152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.230838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2610a6-d3a8-46af-9ae9-7af5a8fedd0d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.266676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.267349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.267578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.267857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.268481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.269847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:46.281150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:46.282162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:46.283009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 378/760] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:46.283804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.294532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.309650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88864fae-543a-4584-82ac-011afc89d9ab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.347190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.347951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.348259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.348847 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.351462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.391146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8a1e6-651e-4b22-a3d9-d35dc1f58783 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.428039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.429551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.429966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.430580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.430855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.431125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.431825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:46.432189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.433649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.446409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 379/761] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:46.449609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.524890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.540127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 383/762] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:52:46.540576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.578454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb101842-bcf7-4cae-b2bb-c13ed1505263 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.604815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.618626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.619242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.619521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.619796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.620561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=10831ea9480d488ab2a81bece14ff090) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:46.620868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.622325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.632053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 380/763] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/role_assignments?user.id=10831ea9480d488ab2a81bece14ff090&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:46.632053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.658309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5ce44-4f4b-459e-b74c-75811cace250 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.700028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.700942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.701403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.702550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.703866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:46.704361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.706519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.713632 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 384/764] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:46.715214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.774502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd78eea-03fe-4dc5-8d01-12cf61ca923f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.786626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.815452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.816151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.816445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.816723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/508f557ac5e34c67a31e4d55f8142a0f/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.819445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=508f557ac5e34c67a31e4d55f8142a0f, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.821637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.842977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca472bc-fdb3-4015-8db9-0e136e9d6ab8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:46.848108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 381/765] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:46 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/508f557ac5e34c67a31e4d55f8142a0f/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:46.850228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:46.922921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:46.923638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:46.923925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:46.924274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:46.945650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:46.947153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:46.952538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:46.979837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:46.980987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:46.981981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:46.981981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:46.981981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:46.988268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 385/766] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:46.991898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.060969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c323f-891c-4ccf-9860-1415616ab287 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.108286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.109281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.109726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.110160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.111225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:47.111720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.114066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a49461-c420-49c0-a42a-941b18c81617 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.122286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 382/767] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:47.124932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c0e85-9da3-41a4-b4b9-a16c3b3e21a1 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.125396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c0e85-9da3-41a4-b4b9-a16c3b3e21a1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.125845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c0e85-9da3-41a4-b4b9-a16c3b3e21a1 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.128414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 383/768] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:47.133534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c0e85-9da3-41a4-b4b9-a16c3b3e21a1 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.151344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc416ef-e18b-42f3-9f6c-7534f804b65d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.155063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:47.191348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.192352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.192774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.193194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.194010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:47.194461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.196226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d2af9-3012-49a6-8b88-fa5cf4161b3b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.210286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 386/769] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:47.212096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.212557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.212982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.232890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:47.234485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:47.259625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c0e85-9da3-41a4-b4b9-a16c3b3e21a1 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.275556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:47.302185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.302994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.303419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.303804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.304829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] RBAC: Adding query filter params (interface=public, service_id=f16b668f51ff404e8b948cb9ba623fbf, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:47.305239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.306857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.320182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 384/770] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/endpoints?service_id=f16b668f51ff404e8b948cb9ba623fbf&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:47.321408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.322828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 387/771] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:47.325174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.514485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:47.525114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214eda-a41b-4dd3-8c01-d59b4c5ec4e8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.538847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59927bf1-4e91-4397-b333-8304c9d7269b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.597599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.599390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.599898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.600663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.601946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.602790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.603567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.605125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.605556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.605980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.618123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 388/772] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/services => generated 772 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:47.618629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.618992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.620739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:47.644194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:47.645179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:47.646065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:47.654154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 385/773] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:47.659068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.741500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:47.786532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6fdef-076c-49f4-8c8c-11b7b0fc527c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.825078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.825713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.825996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.826268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.826901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.828320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:47.838516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:47.840062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:47.841098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 389/774] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:47.847012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf220f7-f4da-4f94-82e4-2b3b61f80d84 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:47.848210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:47.875614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:47.894785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:47.895681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:47.896142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:47.897315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:47.898276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:47.898740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:47.901515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:47.927140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 386/775] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:47.927663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:48.000953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786196bc-f6b4-4e3f-a4b3-1fcbca11b5b6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:48.081292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:48.083258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.085106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.085598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.086530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] RBAC: Adding query filter params (name=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:48.087183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:48.090013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:48.099827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 390/776] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:48.105048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:48.111492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:48.122497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097fbf0-8843-4898-ab61-4775dacc2b27 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:48.189184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:48.189533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.189768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.190049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.190817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=938980e29ada4c4eba2a4172845396e1) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:48.191134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:48.192685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:48.208454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:48.208650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 387/777] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:47 2022] GET /identity/v3/role_assignments?user.id=938980e29ada4c4eba2a4172845396e1&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:48.234783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2bd47-c18e-40ec-8269-6e059c5e8eaf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:48.275688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:48.276674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.277182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.277660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.278519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:48.279025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:48.281530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:48.287693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 391/778] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:52:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:48.293221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:48.448528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b4da6-9987-4d67-9b96-385ae0372094 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:48.497675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:48.498323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.498610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.498885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.499657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] RBAC: Adding query filter params (interface=public, service_id=e00004945c1144c1b66f5546e0e931b1, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:48.499977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:48.501420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fbe0e9-d729-41b3-990d-c325cf9ad03b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:48.512148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 388/779] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:52:48 2022] GET /identity/v3/endpoints?service_id=e00004945c1144c1b66f5546e0e931b1&interface=public®ion_id=RegionOne => generated 194 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:48.518646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:48.652718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755778-f5e5-439e-b02d-b65a947dd1f8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:48.702284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14819d56-282a-4beb-b659-a4f7c886a47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:48.702995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14819d56-282a-4beb-b659-a4f7c886a47f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.703278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14819d56-282a-4beb-b659-a4f7c886a47f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.703554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14819d56-282a-4beb-b659-a4f7c886a47f None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.704194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14819d56-282a-4beb-b659-a4f7c886a47f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:48.705570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14819d56-282a-4beb-b659-a4f7c886a47f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:48.717469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 392/780] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:52:48 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:48.874294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776db2d3-3e87-457c-8ddc-8aea97e1a22c None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.874607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776db2d3-3e87-457c-8ddc-8aea97e1a22c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.874834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776db2d3-3e87-457c-8ddc-8aea97e1a22c None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.876537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 389/781] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:48.894069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277efd-a8d6-4257-866d-08fce9aebc82 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:48.894403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277efd-a8d6-4257-866d-08fce9aebc82 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:48.894624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e277efd-a8d6-4257-866d-08fce9aebc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:48.926106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8e277efd-a8d6-4257-866d-08fce9aebc82 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:48.927333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8e277efd-a8d6-4257-866d-08fce9aebc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:48.940477 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:49.013329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 393/782] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:49.330127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b832b-1aae-427f-ac02-5450a48d83b6 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:49.330642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b832b-1aae-427f-ac02-5450a48d83b6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:49.330642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b832b-1aae-427f-ac02-5450a48d83b6 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:49.332111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 390/783] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:49.337602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:49.337891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:49.338167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:49.364981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:49.366175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:49.383908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b832b-1aae-427f-ac02-5450a48d83b6 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:49.420875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:49.480640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 394/784] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:49.544696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b832b-1aae-427f-ac02-5450a48d83b6 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:49.598048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:49.598601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:49.598982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:49.599264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:49.620901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:49.626271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:49.627205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:49.628085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:49.630440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 391/785] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:49.635702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:49.758550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:49.795877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed054479-7e8e-48a9-856a-f0be4dafd1ff None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:49.813453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:49.847907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:49.848642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:49.848900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:49.849156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:49.849982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:49.850497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:49.855605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:49.879068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 395/786] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:49.883166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:49.959741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:49.979044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31a5a2-9530-4852-853f-1ddaee6c9018 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.020443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.021379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.021669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.021948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.034033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:50.039579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:50.040555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:50.041412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 392/787] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:50.048794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.111542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09067a-ab89-4278-9e0c-78610ec05caa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.167733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96982be0-979a-4620-b9c1-a44bcf47ce82 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.208033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.208033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.208033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.208033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.208455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:50.208455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.208455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.216193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.217286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.217557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 393/788] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:50.218143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.218430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.228049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.229678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:50.243473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:50.250865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.252371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:50.278144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:50.279155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:50.280493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:50.280493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:50.280493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:50.280493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 396/789] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:50.286899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.382654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e711675e-aea7-459f-a5b5-2ccd13676a34 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.407048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422a8b2-71c8-41b8-90e5-66161f440f77 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.426763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.427440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.428192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.428483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.442291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.442819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.443432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.443722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.443892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.444076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.444670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:50.444977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.446566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc2c19-536d-4331-acb6-9614d7ffb32f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.459845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 397/790] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:50.461235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9e7db-518b-4f05-89c2-05b82739c1f4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.461521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9e7db-518b-4f05-89c2-05b82739c1f4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4910c99a-ea35-4231-88f6-18689232198d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:50.461727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:50.463561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:50.464993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:50.464993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:50.464993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:50.465192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9e7db-518b-4f05-89c2-05b82739c1f4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.465277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 394/791] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:50.465358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 398/792] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:50.470178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.470178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.470178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.471984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9e7db-518b-4f05-89c2-05b82739c1f4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.500199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:50.502020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:50.534282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:50.572224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:50.600031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 395/793] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:50.600031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.617309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9e7db-518b-4f05-89c2-05b82739c1f4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.682308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.683096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.685092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.685092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.685092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:50.685092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.686732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaaeeb6-26db-4070-87fa-7734f0ac6057 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.705234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 399/794] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:50.705234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9613a7-31f8-4f17-8be9-fcab191f4dfc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.705234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9613a7-31f8-4f17-8be9-fcab191f4dfc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.705234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9613a7-31f8-4f17-8be9-fcab191f4dfc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.705234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 400/795] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:50.705234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9613a7-31f8-4f17-8be9-fcab191f4dfc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.780157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d70de-7b91-4eb1-bf1e-e1c2f8587694 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.821893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.822823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.823187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.823571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.832211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.833926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.843667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9613a7-31f8-4f17-8be9-fcab191f4dfc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-506666ff-db30-419c-8f7a-078023da3f1f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:50.848476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:50.849592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:50.850789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:50.852726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 396/796] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:50.856773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:50.859745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.860420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.860981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.882139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:50.883903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:50.888771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:50.889788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:50.890081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:50.890359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:50.899819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:50.901603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:50.919379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:50.921145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:52:50.922109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:50.925830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 401/797] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:50.926560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:50.938219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:50.968084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 397/798] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:50.968084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.048870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ddeda-9cab-4ef9-8d84-5437ba55e233 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.071027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51da0a3-72d6-4b0e-a95e-679e9898c780 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.085204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.085812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.086094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.086371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.087214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:51.087563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.089396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.108782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.109408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.109689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.109959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.110548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.111974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.114651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 402/799] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:51.115573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-2927ddb7-0f47-4384-890b-8f3774558d1a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:51.121346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:51.122321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:51.123197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 398/800] 10.222.0.20 () {56 vars in 1129 bytes} [Wed Aug 31 10:52:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:51.124144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb02df-a7f9-4e93-a0bd-15010d3561fd None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.124214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb02df-a7f9-4e93-a0bd-15010d3561fd None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.126131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb02df-a7f9-4e93-a0bd-15010d3561fd None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.129593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 399/801] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:51.130374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb02df-a7f9-4e93-a0bd-15010d3561fd None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.153243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.183191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.239449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654f5ec-fbfe-4edf-b869-484cdf03d3ee None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.253102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb02df-a7f9-4e93-a0bd-15010d3561fd None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.274646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.275261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.275551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.275812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.276592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:51.276903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.278416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.287435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.288087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.288366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.288643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.296588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 403/802] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:51.297432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.303822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.305315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5e31921f-5ac3-4aa7-889c-ef85c00f1947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:51.321370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:51.322409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:51.323328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:51.323328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:51.323328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:51.323328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 400/803] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:51.323711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.323904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.324202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.343916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:51.345290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:51.402800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.423335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 401/804] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:51.424834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.429993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd800f6-12be-4ae9-aca7-6988a6c5e834 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.462466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.469106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.469719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.469998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.470269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.470919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:51.471222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.472644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.482160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 404/805] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:51.482160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.552250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb537e-f644-4bfc-862e-2d838e2c9db6 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.593516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.594610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.595119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.595601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.601258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673bfef-852c-470c-b14a-242153cd53bb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.607087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:51.612072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:51.613090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:51.613923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 402/806] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:51.614628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.642180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.643125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.643608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.644107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.665447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.714996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.728456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.729937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.734828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dd688-773b-48a9-8f77-84bcf5fc274a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.770429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.771039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.771329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.771609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.772290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:51.772600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.774044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.787638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 403/807] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:51.788668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.843903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 405/808] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:52:51.844550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.893713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302dbf2-a3e6-4a5c-809e-5373a9d19967 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.894554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.928210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.928815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.929098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.929375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.930012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:51.930322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.931697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.942360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 404/809] 10.222.0.20 () {56 vars in 1124 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:51.942360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:51.946183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7454c2-e516-4378-875d-0d938480804c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:51.954323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:51.984060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:51.984995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:51.985270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:51.985542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:51.986207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:51.986514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:51.987885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:51.996102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 406/810] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:51.996930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.046197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd3a079-ff7d-4da5-8056-2fc44d409c95 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.082926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.083572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.083854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.084162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] PATH_INFO: `/v3/services/network` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.084756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.086138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:52.096321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:52.097559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 10:52:52.098782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:52.102433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 405/811] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:52.103221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.113038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61637448-4275-41d7-bd0c-74b7840bee2a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.123990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.149567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.150211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.150491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.150763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.159126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.160630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:52.172702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:52.173856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 407/812] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:52.175128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.179126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.210714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5f9f0-1bd4-4bce-bce9-06cb2a3f709f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.246344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.246977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.247258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.247532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.260755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.262236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:52.277695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:52.278738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:52.280568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:52.280568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:52.280568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:52.280568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 406/813] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:52.280568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.285716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362cf8c0-11f2-45ef-854b-0af8aedc5c7f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.320914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.321533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.321814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.322087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.322762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.324213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.349468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 408/814] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:52:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:52.350799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.393637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.399866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14dcd8-bbd9-42cc-b4af-8a9c514b7879 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.431378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.450755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.452133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.452486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.452830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.466552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.468427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:52.485311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:52.486414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:52.487327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:52.487327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:52.487327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:52.491708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 407/815] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:52.492690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.524712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e9e1f-7b92-46db-a5b8-e6ac30394aab None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.568215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.569135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.569539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.569821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.570460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] RBAC: Adding query filter params (name=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:52.570770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.572201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.577163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 409/816] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:52.577814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.614954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75e483-3444-4e0f-a835-1ad3a70fd753 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.651463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.652090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.652369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.652637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.653318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:52.653622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.655615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.669224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 408/817] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:52.673166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.681284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.692398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f902d-46a5-462e-842a-a8d8c20b241f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.706443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.732108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.732816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.733006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.733275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.733921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:52.734221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.735710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.752584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 410/818] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:52.752584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.799778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe6ad8-ed49-42e6-8cfe-4c5ee23ab4c2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.845818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.846436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.846831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.847108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.847760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:52.848139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.849490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.866726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 409/819] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:52.866726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.869479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1489e0bc-8196-45e0-8ab5-0595227fc02c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:52.919272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:52.920461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:52.920867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:52.921337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:52.922849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] RBAC: Adding query filter params (type=network) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:52.922849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:52.924919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:52.933406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 411/820] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:52.934272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:52.952778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.975953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:52.986528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc56f4-19cf-4b42-943e-1233e6b10806 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.023853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.024556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.024837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.025114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/10831ea9480d488ab2a81bece14ff090/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.027119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=10831ea9480d488ab2a81bece14ff090, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.028610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.056056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 410/821] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:52 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/10831ea9480d488ab2a81bece14ff090/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:53.056056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.056364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc749fc5-95ff-43c7-b6fa-25188e8aab74 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.097100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.098001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.098287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.098563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.106921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.109443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:53.122914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:53.123972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:53.124913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 412/822] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:53.125413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.200800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ad1c6-3556-449c-9cbf-84e553941a74 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.221373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:53.251728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.252435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.252715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.252988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.254499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0123b7-1843-4738-874c-6acb31503859 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.264689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.267665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:53.285618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:53.286636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:52:53.287591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:53.294146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 411/823] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:52:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:53.295144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.297396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.298638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.300246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.300733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.301609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.303208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.311961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:53.335010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 413/824] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:52:53 2022] POST /identity/v3/endpoints => generated 337 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:53.336234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.475664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13e1a8-881b-4f58-8989-314548c04dc7 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.506533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:53.518276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.518895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.519174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.519278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9e8f6-0a85-4c37-b278-5a7e6ad2efba None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.519687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.520210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:53.520522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.522117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.542840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 414/825] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 10:52:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:53.553782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.575801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.576983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.577549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.578022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.578926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:53.579469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.581233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.599158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 412/826] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:53.605090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.708926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6481f-2047-43f4-9bde-7cabe23f002c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.721020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:53.753364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.757072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.757624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.758106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.769371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a794-235a-467d-b993-6af6b5761bac None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.770220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.771846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:53.797271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:53.798274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:53.799176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:53.799176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:53.799176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:53.799176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 415/827] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:53.804942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:53.826646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.827864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.828358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.829052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.830368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=508f557ac5e34c67a31e4d55f8142a0f) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:53.830906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.833180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:53.842669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 413/828] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:53 2022] GET /identity/v3/role_assignments?user.id=508f557ac5e34c67a31e4d55f8142a0f&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:53.852987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:53.939625 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2ce71-67a7-4af5-be77-6676d48cbf9e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:53.981001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:53.982025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:53.983079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:53.983555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:53.984443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:53.984970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:53.986738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:54.001568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 416/829] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:54.008020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:54.125403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fe70bf-540b-440c-9d9f-f6df8f224814 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:54.164630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:54.165267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:54.165547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:54.165820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:54.168719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:54.173476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:54.175026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:54.188884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:54.189924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:54.190825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:54.194827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 414/830] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:54.196282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:54.298727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:54.305251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b431d-062b-4561-adc3-67ea72091a68 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:54.341939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:54.342691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:54.343037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:54.343381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:54.344182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:54.344560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:54.346119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbd7ed-6305-436e-9266-29d3ecfaeeb0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:54.359065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 417/831] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:54.364973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:54.472770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b397af-ec71-4deb-9890-9ac32cae711b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:54.511604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21050f-b2cc-4e0b-8f42-aa9dcca8983d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:54.513832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8db181-b658-4aa4-bda4-57fd80e49e2e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:54.514192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8db181-b658-4aa4-bda4-57fd80e49e2e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:54.514540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8db181-b658-4aa4-bda4-57fd80e49e2e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:54.516107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 418/832] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:54.516469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21050f-b2cc-4e0b-8f42-aa9dcca8983d None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:54.516819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21050f-b2cc-4e0b-8f42-aa9dcca8983d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:54.517160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21050f-b2cc-4e0b-8f42-aa9dcca8983d None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/938980e29ada4c4eba2a4172845396e1/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:54.523968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:54.532052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:54.532052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:54.545735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:54.547335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:54.647099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 419/833] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:54.670917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21050f-b2cc-4e0b-8f42-aa9dcca8983d None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=938980e29ada4c4eba2a4172845396e1, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:54.675766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21050f-b2cc-4e0b-8f42-aa9dcca8983d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:54.699882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 415/834] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:52:54 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/938980e29ada4c4eba2a4172845396e1/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:52:54.935317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:55.037602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:55.053253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff08cbc-bd59-4aec-ac00-1263efcff9c7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:55.091208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:55.092709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.093127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.093402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.093992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:55.095359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:55.099532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d6834-838f-47f2-8449-fed0ea594814 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.099820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d6834-838f-47f2-8449-fed0ea594814 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.100121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1d6834-838f-47f2-8449-fed0ea594814 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.101728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 416/835] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:55.104482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:55.105435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:55.106268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 420/836] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:55.107062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.107375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.107671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.114936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:55.128417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:55.129628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:55.189404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:55.236901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 417/837] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:55 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:55.243908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f3068-7871-42bc-a834-cab3877af85f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:55.283280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:55.284252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.284760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.285226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.286059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:55.286552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:55.288337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:55.294497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 421/838] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:55.300734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:55.413585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c94c-bfae-4649-9878-9ed9a9343734 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:55.450202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:55.451672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.452205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.452725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.453543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:55.454024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:55.455572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:55.460976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 418/839] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:52:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:55.519017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:55.542074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:55.642147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a6f1d-eb08-4084-9e4b-4b2b714a0f2a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:55.678224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:55.678861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.679142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.679417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.691229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:55.697640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:55.698688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:52:55.699539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:55.704315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 422/840] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:55.706429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:55.754851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:55.817870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f6b2e-78e3-4c32-8957-198951fbfee9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:55.853878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:55.854490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:55.854771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:55.855047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:55.855721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:55.856041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:55.858171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:55.872191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 419/841] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:52:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:55.874237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:55.958202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:55.985762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a56b1-bcfc-4b97-bd68-9fefe71470f0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.021584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.022235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.022515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.022789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.035157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.036721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8397a2ff-872f-4f80-a4b3-9beb8cda4421 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:56.053411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:56.054436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:56.055343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:56.055343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:56.055343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:56.059525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 423/842] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:56.061779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.082222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15bbb-a815-4c22-8115-9a6dc68296aa None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.082504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15bbb-a815-4c22-8115-9a6dc68296aa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.082777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f15bbb-a815-4c22-8115-9a6dc68296aa None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.084936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 424/843] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:52:56.089766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.090045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.090318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.111169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:56.112390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:56.191541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 425/844] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:52:56.196842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f110fc9-913e-4346-a7f5-662665675fdf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.198625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:56.236155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.236693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.236970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.237269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.237894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:56.238197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.239572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.256914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 420/845] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:56.258254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.321464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:56.369659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312e612-af6f-4880-83ac-9ff07a121189 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.406150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.406791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.407069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.407346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.415009 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.416486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:56.429424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:56.430636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 10:52:56.431790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:56.435449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 426/846] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:56.436948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.487432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.559720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999543f-b061-42c1-b77a-9951bfa77681 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.603873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.604386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.604666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.604937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.605673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:56.605979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.606427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704e649-995b-4db7-87d1-fda4f11749b4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.607650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.630338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:56.636103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 421/847] 10.222.0.20 () {56 vars in 1135 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:56.641784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.657903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.658990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.659480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.660076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.660945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.662783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.671259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:56.675384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:56.676878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:52:56.677799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:56.684545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 427/848] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:52:56.688275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.762849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781d82e-94ad-4d72-a51f-41f24f23c8eb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.806701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06258e33-976c-4e5f-9979-59a5783c18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.807201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258e33-976c-4e5f-9979-59a5783c18a6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.807480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258e33-976c-4e5f-9979-59a5783c18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.807745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06258e33-976c-4e5f-9979-59a5783c18a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.817434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bcc163-8aba-4836-babe-0ea58557df62 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:56.854809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:56.855399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.855678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.855952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.856618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:56.856926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.858337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30775f8d-c38d-4e87-81ed-5f27af3bae21 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.864060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 428/849] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:56.864514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863434d-3bc9-48b4-8b83-f07a6e9be303 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:56.864799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863434d-3bc9-48b4-8b83-f07a6e9be303 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:56.865071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863434d-3bc9-48b4-8b83-f07a6e9be303 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:56.870212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 429/850] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:56.870212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863434d-3bc9-48b4-8b83-f07a6e9be303 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:56.889440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06258e33-976c-4e5f-9979-59a5783c18a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:56.890882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06258e33-976c-4e5f-9979-59a5783c18a6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:56.914064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:56.923113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:56.991776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863434d-3bc9-48b4-8b83-f07a6e9be303 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.006735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 422/851] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:52:57.006735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.006735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.006735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.024681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:57.025802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:57.028945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.029617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.029901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.030174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.030810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:57.031113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.032537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.037340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 430/852] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:52:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:57.038154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.131527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 423/853] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:57.160993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84f77b7-5c57-41c9-91d8-72fa0af0674f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.172566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:57.202658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.203304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.203581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.203906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.217648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.219101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:57.235925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:57.236997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:57.238540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:57.238540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:57.238540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:57.238540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 431/854] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:57.242620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.298741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:57.357955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47831a-7751-4f14-9e75-9deac49b11fe None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.394922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.395535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.395814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.396110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.396749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:57.397045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.398465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.411790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 424/855] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:57.417120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.449292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.529740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea2c04-be97-4d68-ad2e-65df4d5fbe83 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.569663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.570216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.570506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.570777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.577376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3e5b-6dbd-451e-8934-3d8b31a68d9c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.578184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.579648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.589461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-24d918e4-a7fe-4108-a555-8a48d4ca0a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:57.592320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:57.593239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 432/856] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e18df4-6227-4e1c-acc4-7e8ed60f9fa3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.599454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e18df4-6227-4e1c-acc4-7e8ed60f9fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.600081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e18df4-6227-4e1c-acc4-7e8ed60f9fa3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.601097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 433/857] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:57.601937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e18df4-6227-4e1c-acc4-7e8ed60f9fa3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.627527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06db910-8f56-4210-b3bf-3a61d0cfcb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.628234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:57.628762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db910-8f56-4210-b3bf-3a61d0cfcb8c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.629043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db910-8f56-4210-b3bf-3a61d0cfcb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.629316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06db910-8f56-4210-b3bf-3a61d0cfcb8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.642042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06db910-8f56-4210-b3bf-3a61d0cfcb8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-d06db910-8f56-4210-b3bf-3a61d0cfcb8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:52:57.648316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:57.649281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:52:57.650146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:57.654891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 425/858] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:57.655943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.656257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.656523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.674590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:57.675781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:57.720028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e18df4-6227-4e1c-acc4-7e8ed60f9fa3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.749221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 426/859] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:57.750370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.768285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.768893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.769170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.769443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.770135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:57.770437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.771893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.792655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 434/860] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:57.799116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.897914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:57.905267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd29e92e-d71e-4b6c-aab6-53d68224e3a1 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.910611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:57.933750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4967d-419d-44f4-a1b0-d7578063293a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:57.956886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.957766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.958459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.958886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.959978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:57.960508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.963083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.973474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:57.974240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:57.974613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:57.974979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:57.975797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=10831ea9480d488ab2a81bece14ff090) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:57.976244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:57.977126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 427/861] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:57.981156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:57.982258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:57.990735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 435/862] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/role_assignments?user.id=10831ea9480d488ab2a81bece14ff090&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:58.040800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:58.117155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158b4cd-f088-4351-8b5e-de9d410b91b7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:58.163211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:58.163844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:58.164160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:58.164667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:58.167696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf15da0-6fca-4d64-9b20-68b56fb208eb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:58.177330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:58.178789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:58.195673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:58.196752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:58.197816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:58.197816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:58.197816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:58.199672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 428/863] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:58.207261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:58.219541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:58.220200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:58.220485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:58.221124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:58.221816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:58.223849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:58.229148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:58.246372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 436/864] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:52:58 2022] POST /identity/v3/services => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:52:58.337250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade4364-dcb3-472d-8071-ae2872cd1e10 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:58.390091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:58.391027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:58.391450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:58.391888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:58.392700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:58.393160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:58.394672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:58.408711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 429/865] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:58.414286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:58.421190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:58.531537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863291c-2809-411f-9e17-933ebce7c620 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:58.568143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:58.568854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:58.569132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:58.569405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:58.578273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:58.579748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:52:58.592505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:58.593718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:58.594875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 437/866] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:52:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:58.599769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:58.678960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:58.714465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2339e4-a212-40f1-a3ff-7b7d09ad7c90 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:58.753524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:58.754384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:58.754814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:58.755268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:58.756177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:58.756631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:58.758198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cb1e91-d98a-4d0a-94bc-222dc06b532d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:58.777289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 430/867] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 10:52:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:58.784881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:58.853302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:58.894715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c9cf-2f07-4830-aed7-a89011db30be None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:58.930521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:58.931047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:58.931323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:58.931591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.009534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:59.011008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:59.073582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605454-f96d-4576-aa0e-3965cabfd320 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.074101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605454-f96d-4576-aa0e-3965cabfd320 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.074101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72605454-f96d-4576-aa0e-3965cabfd320 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.076414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:59.081269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 431/868] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:52:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:52:59.083043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bf760-485c-4187-a227-f089e9c841be None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.083327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bf760-485c-4187-a227-f089e9c841be None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.083598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bf760-485c-4187-a227-f089e9c841be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.103410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-090bf760-485c-4187-a227-f089e9c841be None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:52:59.104792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-090bf760-485c-4187-a227-f089e9c841be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:52:59.131728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 438/869] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:52:58 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:52:59.133020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:59.192062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 432/870] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:52:59 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:52:59.243751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b45bd-7271-474a-ad79-c69ab1252892 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:59.282968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:59.283942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.284455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.284911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.296318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:59.298054 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:52:59.324109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:59.325127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:52:59.327476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:52:59.327476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:52:59.327476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:52:59.329352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 439/871] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:52:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:59.337167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bf760-485c-4187-a227-f089e9c841be None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:59.403775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:59.447873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bf760-485c-4187-a227-f089e9c841be None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:59.475090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:59.488392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:59.489023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.489304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.489578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.490313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:59.490620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:59.492729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:59.499016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:59.514749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 433/872] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:52:59.516445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:59.598355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa793b7-7bde-45ca-8a75-e52133b51cdb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:59.643519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed6d4b-2d80-47b1-865d-6b4c7c89f7c4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:59.643835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:59.644464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.644745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.645020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.645697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:59.648284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:59.674057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:59.674406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 440/873] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:52:59 2022] POST /identity/v3/services => generated 269 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:52:59.698167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:59.699684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.700327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.701365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.710703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:59.712549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:52:59.726648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:52:59.727621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:52:59.728565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:52:59.731339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 434/874] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:52:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:52:59.737031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:52:59.838923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:52:59.870135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe05a5-bc9a-4ce8-9a05-c119f54dbb8e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:52:59.908177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:52:59.908795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:52:59.909073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:52:59.909345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:52:59.910045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:52:59.910348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:52:59.911790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dec04bd-3ee0-4914-9e9d-33c7fa427663 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:52:59.926144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 441/875] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:52:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:52:59.932124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:00.042693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c96cae-ce69-44c9-a022-db0569fcaac3 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:00.084403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:00.085927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.086409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.086878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.087829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=938980e29ada4c4eba2a4172845396e1) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:00.088417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:00.090327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467646c-99d1-4536-9689-c29bf5dce6ec None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:00.097866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 435/876] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:52:59 2022] GET /identity/v3/role_assignments?user.id=938980e29ada4c4eba2a4172845396e1&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:00.110270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ba41d-f924-4850-893f-721a3c4538f6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.110824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ba41d-f924-4850-893f-721a3c4538f6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.110824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ba41d-f924-4850-893f-721a3c4538f6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.112468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 442/877] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:00.121694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c05c0-36b5-4b8f-8ebe-dcedc3938597 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.122021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c05c0-36b5-4b8f-8ebe-dcedc3938597 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.122250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c05c0-36b5-4b8f-8ebe-dcedc3938597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.142638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-cd2c05c0-36b5-4b8f-8ebe-dcedc3938597 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:00.143900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cd2c05c0-36b5-4b8f-8ebe-dcedc3938597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:00.180144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:00.226541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 436/878] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:00.401602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81766-fcc7-4ce0-a514-f57c34cb906b None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.401798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81766-fcc7-4ce0-a514-f57c34cb906b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.402074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b81766-fcc7-4ce0-a514-f57c34cb906b None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.403651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 443/879] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:00.409869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.410142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.410415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.429223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:00.430402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:00.501292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81766-fcc7-4ce0-a514-f57c34cb906b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:00.502541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 437/880] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:00 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:00.615127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b81766-fcc7-4ce0-a514-f57c34cb906b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:00.652110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:00.652671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.652947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.653221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.654394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:00.655852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:00.658815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:00.667411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:00.668525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:00.670051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 444/881] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:00.677899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:00.788267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:00.795676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:00.805534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e0307-f58b-45db-9a1c-fe5a1805f542 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:00.843443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:00.844596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.845070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.845577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.846402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:00.846994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:00.848728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:00.854466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 438/882] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:00.861565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:00.921300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325d6ab-7b9e-4375-9e1f-2439c06dd954 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:00.971742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:00.972851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:00.973344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:00.973938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:00.974748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:00.976389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:00.987240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:00.988299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:53:00.989123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:00.989850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c5147-fca7-4b68-9276-406f4a9ce25d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:00.992165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 445/883] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:53:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:00.997811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:01.043044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:01.043655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.043933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.044294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.044960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:01.045310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:01.046945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:01.057035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 439/884] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:01.082696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:01.127808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d19caf-8f7f-45bd-bf86-6ccab8855954 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:01.168616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:01.170123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.170553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.171149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.172077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:01.172527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:01.174150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:01.180221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 446/885] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:53:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:01.188363 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:01.218410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:01.302928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd540182-fac4-4442-a663-5ffd506f4cf5 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:01.339948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:01.340591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.340866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.341137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.342537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:01.342843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:01.344274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:01.350388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 440/886] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:53:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:01.358479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:01.465731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c671-5885-4606-a4bd-51c4b44abc1d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:01.500032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:01.503666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:01.504635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.505124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.505590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.506579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] RBAC: Adding query filter params (interface=public, service_id=b8e1d4ba1b6f48efb1fb8ba7147943d3, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:01.507193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:01.508870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6fddaa-1ba6-4c53-8d21-6870787bf0e0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:01.517705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 447/887] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:53:01 2022] GET /identity/v3/endpoints?service_id=b8e1d4ba1b6f48efb1fb8ba7147943d3&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:01.526097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:01.629610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:01.641768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37196085-2961-4720-966f-d5b5ededd7b7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:01.678143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7f043-499d-49d9-b201-588e21afbbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:01.678752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f043-499d-49d9-b201-588e21afbbc5 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.679028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f043-499d-49d9-b201-588e21afbbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.679345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7f043-499d-49d9-b201-588e21afbbc5 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.679954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f043-499d-49d9-b201-588e21afbbc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:01.681514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7f043-499d-49d9-b201-588e21afbbc5 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:01.689204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 441/888] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:53:01 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:53:01.910656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458ef96-a08f-44dd-bb89-bf8458e9df90 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.911040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458ef96-a08f-44dd-bb89-bf8458e9df90 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.911220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c458ef96-a08f-44dd-bb89-bf8458e9df90 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.912884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 448/889] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:01.918166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518b091-1dae-4733-bbf9-5a05c7a4a290 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:01.918458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518b091-1dae-4733-bbf9-5a05c7a4a290 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:01.918736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518b091-1dae-4733-bbf9-5a05c7a4a290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:01.938180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4518b091-1dae-4733-bbf9-5a05c7a4a290 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:01.939382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4518b091-1dae-4733-bbf9-5a05c7a4a290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:02.017558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 442/890] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:01 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:02.136565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6a2e-4c46-4239-addf-47c6cbdc0935 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.137528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6a2e-4c46-4239-addf-47c6cbdc0935 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.138093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d6a2e-4c46-4239-addf-47c6cbdc0935 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.140954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 449/891] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:02.146193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.146759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.147233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.167024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:02.168580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:02.244470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 443/892] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:02.297708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d6a2e-4c46-4239-addf-47c6cbdc0935 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:02.407066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d6a2e-4c46-4239-addf-47c6cbdc0935 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:02.443072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:02.443975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.444287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.444562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.445430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:02.446856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:02.457292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:02.458332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:02.459244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 450/893] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:02.465261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:02.481498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:02.548347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:02.582099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a45693-6748-49cc-b0b9-58548d4ff0ce None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:02.627302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:02.627702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:02.628496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.628677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.628923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.630100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:02.630462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:02.632493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:02.640256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 444/894] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:02.645788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:02.693249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd802426-acb1-4c53-b501-ebac4698861b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:02.734592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:02.746755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:02.747821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.748327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.748772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.760228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4bed-5bd7-4aaf-ac92-5133b539fb36 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:02.761020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:02.767070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:02.768057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:02.768913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:02.771705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 451/895] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:02.780082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:02.814196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:02.814814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.815090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.815359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.815993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:02.816317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:02.817667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:02.826519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 445/896] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:02.833271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:02.915607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccd2fac-bcb2-44f9-a504-0db364a31936 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:02.934181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:02.963190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:02.964712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:02.965655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:02.966610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:02.967718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:02.968258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:02.970144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:02.974928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a263e-d5d1-40fd-adee-3e02833d5846 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:02.987224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 452/897] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:02.988981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.018188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.018867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.019155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.019758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.020575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] RBAC: Adding query filter params (interface=public, service_id=6059c2ae21754204b9ca21fe9b265d3d, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:03.020891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.022854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.032044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 446/898] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/endpoints?service_id=6059c2ae21754204b9ca21fe9b265d3d&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:03.037587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.049296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:03.114808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b7fc7-4072-46f2-9e57-278e83946ce1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:03.164541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.166147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.166653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.167196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.173400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39df342-dcdb-4ff9-9f2a-4aec5e0832b9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:03.179360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.181042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-38e1a69a-88c7-411b-8a21-c03bb9bda037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:03.198464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:03.200391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 453/899] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff49e-1c89-44b1-948f-abfa2151ea30 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff49e-1c89-44b1-948f-abfa2151ea30 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.202428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ff49e-1c89-44b1-948f-abfa2151ea30 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.207766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 454/900] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:03.207766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ff49e-1c89-44b1-948f-abfa2151ea30 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.227017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddca33d-a2c2-41ca-ba73-ae0c614a9890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.227545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca33d-a2c2-41ca-ba73-ae0c614a9890 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.227830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca33d-a2c2-41ca-ba73-ae0c614a9890 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.228152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddca33d-a2c2-41ca-ba73-ae0c614a9890 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.228765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca33d-a2c2-41ca-ba73-ae0c614a9890 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.230138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddca33d-a2c2-41ca-ba73-ae0c614a9890 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.241847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 447/901] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:53:03 2022] GET /identity/v3/services => generated 1268 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 10:53:03.243178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.243465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.243743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.257953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:03.269150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:03.270406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:03.357857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:03.363928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 448/902] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:03.378433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ff49e-1c89-44b1-948f-abfa2151ea30 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:03.426916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.427557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.427838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.428144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.428792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:03.429097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.430539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.445298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 455/903] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:03.452270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.565618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e5ac9-4739-4e69-9096-ed9c176b56aa None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:03.606243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.606926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.607224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.607480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.615393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.617067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:03.630401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:03.631419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:53:03.632393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:03.636119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 449/904] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:53:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:03.637686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.640184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.714854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:03.751410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:03.764577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780dba3-b641-4d1a-8c60-01247261c9e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:03.775843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041d5b8-dee8-4176-b357-e5d65242f00a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:03.815557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936d97a-1a76-4f46-be39-dd3bd583010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.816282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936d97a-1a76-4f46-be39-dd3bd583010f None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.816538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936d97a-1a76-4f46-be39-dd3bd583010f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.816764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936d97a-1a76-4f46-be39-dd3bd583010f None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.817419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936d97a-1a76-4f46-be39-dd3bd583010f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.818783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936d97a-1a76-4f46-be39-dd3bd583010f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.821642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:03.822420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.822693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.822959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.823706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:03.823999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:03.825493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:03.837325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 456/905] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:53:03 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:03.839227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff147e-5015-4b32-b43c-edd1bc11b686 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.839717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff147e-5015-4b32-b43c-edd1bc11b686 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.840195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff147e-5015-4b32-b43c-edd1bc11b686 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.842362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 457/906] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:03.848332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:03.848840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:03.849251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:03.864169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 450/907] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 10:53:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:03.871023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:03.876715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:03.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:03.992038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 458/908] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:04.022546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce34714-af5d-4155-b26f-2571949ce96a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.043463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.064155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.064778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.065058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.065332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.147872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.150317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.270389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.271191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 451/909] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:03 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:04.277197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.297311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.357581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.385143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a2ca8-4b80-42b8-83b1-5d053abd4270 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.386139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5ca90-be58-4d45-b130-44c72468096d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.425157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.425815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.425968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.426146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.426445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.427174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.427394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.427855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.428376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.428842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:04.438794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:04.439885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 10:53:04.441126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:04.444287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.445960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.446684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 459/910] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:04.448334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:04.470108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:04.471143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:04.472160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:04.472160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:04.472160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:04.476120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 452/911] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:04.477493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.579241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.592213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01e512-205c-444e-8df9-ea93dea2b5b0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.627209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd2ac-8a9b-45d6-af9d-9bcc5672f02a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.633183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.633800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.634073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.634343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.634978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:04.635277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.636682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.642980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 460/912] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:04.648725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.654688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.676757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.678010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.678460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.678888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.679976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:04.680459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.684622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.705807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 453/913] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:04.711035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.800344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748bd00-f585-4f96-8d2c-de42e5a6b408 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.837756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.839050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.839542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.840052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.840905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:04.841710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0a878-d865-4bdf-8226-6c2369e66009 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:04.842236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.844055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.850079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 461/914] 10.222.0.20 () {56 vars in 1114 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:04.857093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.883345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.893412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:04.894597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:04.895070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:04.895708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:04.906275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:04.907936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:04.922621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:04.923576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:04.924542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:04.927465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 454/915] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:04.933683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:04.942437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:04.981825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6355b0b2-94a7-4333-8bd7-ecc96697ff1e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:05.034145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:05.035343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.035831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.036404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.037092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:05.038467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:05.060959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c808e7-ccdd-458d-8846-21864bea9806 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:05.064163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 462/916] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:53:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:05.116857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:05.118027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.118554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.119049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.120078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:05.120716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:05.123029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8df77-d069-4177-9b51-74965ee7ceba None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:05.144073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 455/917] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:53:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:05.145878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:05.261359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678299-365d-4f2f-8e7a-fb48fa3a25d4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:05.288123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:05.300258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:05.300904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.301185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.301461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.302312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] RBAC: Adding query filter params (role.id=bf18b261e4e5416b8ded25432ee11a32, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=938980e29ada4c4eba2a4172845396e1) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:05.302623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:05.304138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ee6da5-b87b-4a83-9963-963b4c2a2c13 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:05.310956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 463/918] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:05 2022] GET /identity/v3/role_assignments?user.id=938980e29ada4c4eba2a4172845396e1&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=bf18b261e4e5416b8ded25432ee11a32 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:05.346395 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc75243-f217-4def-8eb8-293fade2d27b None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.346949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc75243-f217-4def-8eb8-293fade2d27b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.346949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc75243-f217-4def-8eb8-293fade2d27b None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.348618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 456/919] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:05.354412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b45dba-3a1e-4e7a-a623-e02ebff5a9fb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.354718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b45dba-3a1e-4e7a-a623-e02ebff5a9fb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.354974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b45dba-3a1e-4e7a-a623-e02ebff5a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.378583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f1b45dba-3a1e-4e7a-a623-e02ebff5a9fb None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:05.379807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f1b45dba-3a1e-4e7a-a623-e02ebff5a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:05.463743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 464/920] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:05.749519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc75243-f217-4def-8eb8-293fade2d27b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:05.815210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:05.871761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc75243-f217-4def-8eb8-293fade2d27b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:05.908079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:05.909070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.909611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.910098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.910931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:05.912508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:05.921649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:05.922617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 10:53:05.923529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:05.924317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1782-66bc-4fa8-ab31-7b378372d6c0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.924317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1782-66bc-4fa8-ab31-7b378372d6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.924317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be1782-66bc-4fa8-ab31-7b378372d6c0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.925043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 465/921] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:05.926603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 457/922] 10.222.0.20 () {56 vars in 1132 bytes} [Wed Aug 31 10:53:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:05.932028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:05.932028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:05.932028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:05.933176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:05.949351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:05.951290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:06.048744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 466/923] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:06.057023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e988b15-6d70-4a8a-b985-eabc1e04e97a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:06.068711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:06.094883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:06.095761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:06.096245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:06.096858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:06.097668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:06.098188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:06.099807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:06.108979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 458/924] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:06.111485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:06.164751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:06.220848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc7291-acd5-4190-9771-40f1c5d1d519 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:06.259542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:06.260509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:06.260974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:06.261469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:06.262279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:06.262763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:06.265137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:06.275098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 467/925] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:06.276646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:06.323379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:06.388924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeea189-65e7-4572-a986-7b18aa5d175f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:06.427876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:06.428989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:06.429572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:06.430094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:06.431104 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:06.432936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:06.455168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e3101-e1b7-41f2-9675-e1d6d2dad417 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:06.462842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 459/926] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:53:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:06.475070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:06.510296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:06.511483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:06.512069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:06.512656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:06.514342 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:06.516349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:06.527681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:06.529440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:06.530324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:06.533401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 468/927] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:06.540060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:06.582327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:06.663671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abc935-616e-4573-823c-c315827c3c70 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:06.702316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:06.703299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:06.703820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:06.704323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:06.705192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:06.705718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:06.707348 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:06.713193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 460/928] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:06.719826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:06.833193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051992e5-2663-4ea1-a424-412d1fb666da None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:06.870146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:06.870759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:06.871042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:06.871320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:06.872735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:06.873206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:06.874894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:06.880346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 469/929] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:06.886265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:06.992302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:07.000530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f2ba1a-0297-4afd-a9fe-b35072b65558 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:07.036948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:07.037561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.037842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.038121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.038830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] RBAC: Adding query filter params (interface=public, service_id=a7676e80ea4c40c88032e45ac07abeab, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:07.039141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:07.040554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ee4bd-3227-470f-a8cd-1ccba7455ef0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:07.047166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 461/930] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:53:06 2022] GET /identity/v3/endpoints?service_id=a7676e80ea4c40c88032e45ac07abeab&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:07.052734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:07.092468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:07.167792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7ae20-92b7-4a92-a859-d0678b0f8e55 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:07.171498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8b13-7fb9-4507-a1dc-1d7da38bd468 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.171783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8b13-7fb9-4507-a1dc-1d7da38bd468 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.172098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8b13-7fb9-4507-a1dc-1d7da38bd468 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.173696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 462/931] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:07.178781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a71d1-e423-43f9-83ee-9a4f08e7285c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.179056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a71d1-e423-43f9-83ee-9a4f08e7285c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.179329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a71d1-e423-43f9-83ee-9a4f08e7285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.201718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-950a71d1-e423-43f9-83ee-9a4f08e7285c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:07.203409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-950a71d1-e423-43f9-83ee-9a4f08e7285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:07.215861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c33b1-bb1a-4994-81cb-643461889bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:07.217229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c33b1-bb1a-4994-81cb-643461889bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.217520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c33b1-bb1a-4994-81cb-643461889bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.217966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c33b1-bb1a-4994-81cb-643461889bb8 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.218716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c33b1-bb1a-4994-81cb-643461889bb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:07.220417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c33b1-bb1a-4994-81cb-643461889bb8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:07.229130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 470/932] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:53:07 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:53:07.288310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 463/933] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:07.319838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a090d-8557-44c2-b60e-2745c2aca530 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.320172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a090d-8557-44c2-b60e-2745c2aca530 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.320428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a090d-8557-44c2-b60e-2745c2aca530 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.321916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 471/934] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:07.329460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.330035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.330518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.353913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:07.355451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:07.446926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 464/935] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:07.614144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a090d-8557-44c2-b60e-2745c2aca530 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:07.726933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:07.732322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a090d-8557-44c2-b60e-2745c2aca530 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:07.747994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:07.769379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:07.787920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:07.788571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.788846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.789219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.789770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:07.791202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:07.793883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING keystone.server.flask.common [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 10:53:07.813767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.sql.core [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 10:53:07.813767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 10:53:07.813767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [parameters: {'id': '264f52c88fe74344bddb06594e92b53c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 10:53:07.813767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81463) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 264f52c88fe74344bddb06594e92b53c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 264f52c88fe74344bddb06594e92b53c. Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:53:07.821395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 10:53:07.822262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application yield resource Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise self.value Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application session.commit() Aug 31 10:53:07.823692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.session.flush() Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application compat.raise_( Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 10:53:07.824932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 10:53:07.825891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application result.read() Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:53:07.826697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [parameters: {'id': '264f52c88fe74344bddb06594e92b53c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:07.828582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:07.830357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 264f52c88fe74344bddb06594e92b53c. Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 472/936] 10.222.0.20 () {60 vars in 1140 bytes} [Wed Aug 31 10:53:07 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:07.832617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:07.903005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0a384b-9522-4cca-b6c5-0c6191179933 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:07.948446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:07.949491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.949815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c6542-d0b2-4a0b-8d50-c71dbb2eeda8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:07.950225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.950709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:07.965853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:07.972698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:07.973665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:07.974514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:07.977592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 465/937] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 10:53:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:07.985468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:07.992777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:07.993491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:07.993928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:07.994299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.010284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:08.012882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:08.030672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:08.031823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:08.033016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:08.036040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 473/938] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:08.041954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:08.068481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:08.094537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:08.128500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d4ede-4186-41f3-840b-c3e5e263902d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:08.169971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:08.170641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.171012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.171381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.172177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:08.172695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:08.174262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:08.177324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c16aba1-bcdb-4c04-bdae-215a30043532 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:08.183656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 466/939] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:53:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:08.193614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:08.215883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:08.216831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.217263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.217686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.218525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:08.218995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:08.220837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:08.242338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 474/940] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:08.378765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841260d6-1463-4dbf-ae6c-1089af26faae None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:08.429285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:08.429846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.430125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.430397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.441222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:08.444511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:08.446012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c3bce7c9-4b57-4e98-b8b0-a137a353802a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:08.463511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:08.465340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:08.467076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:08.467076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:08.467076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:08.467076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 467/941] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:08.471157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:08.557351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:08.587361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e39d6-df7e-44d0-b816-39c75a69f42e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:08.607846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a20b-be84-46b9-80a4-4fe2355a8200 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.608386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a20b-be84-46b9-80a4-4fe2355a8200 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.608669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75a20b-be84-46b9-80a4-4fe2355a8200 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.610027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 468/942] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:08.615764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.616175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.616525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.636798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:08.637405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:08.638747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:08.639375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.639851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.640330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.641548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:08.642997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:08.644618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:08.658607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 475/943] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:08.664604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:08.748255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 469/944] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:08.808991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1184d-0349-4669-8b09-fe774186290b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:08.851142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:08.851834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:08.852148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:08.853199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:08.856135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:08.862303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:08.863923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:08.877944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:08.879733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:08.881308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 476/945] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:53:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:08.885087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.010581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707a8ed-e4b6-4d65-899c-5c213afdcf52 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.040673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:09.053289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.053906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.054147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.054408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.054408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.055567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:09.055567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.057524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.081900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 470/946] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 10:53:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:09.087510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.175774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe2ec4-adab-4d9c-a551-5901eb0426d2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.205984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6cdd00-5f0d-43ea-8d27-c249a766f2f2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.216742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.217478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.217814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.218165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.218779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.220175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:09.230665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:09.234725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:09.248702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 477/947] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:09.250637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.250781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.250781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.250781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.250781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.262932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.264701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:09.282114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:09.284613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:09.286214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:09.286214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:09.286214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:09.291814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 471/948] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:09.292923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.308731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:09.333744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:09.361234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6d6a-81a8-467e-abcf-b7693bdc549f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.401676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.402627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.403074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.404070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.405029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:09.405481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.407091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.417981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 478/949] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:09.419806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.422731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c8455-dcaf-43ce-a88c-a4b97b236148 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.462159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.462835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.463122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.463657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.464368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:09.464672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.466602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.485090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 472/950] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:09.486090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.537985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c9696-deae-4f64-bef0-d310b3951a41 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.566138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:09.579025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.579650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.579934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.580244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.580885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:09.581193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.582566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234a33b-548d-44eb-bc06-c186054ad51a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.587126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 479/951] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:09.588695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042044c-a9e5-4729-aaca-f2d00b12c91e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.588978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042044c-a9e5-4729-aaca-f2d00b12c91e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.589257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042044c-a9e5-4729-aaca-f2d00b12c91e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.590690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 480/952] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:09.597288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.597806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.598225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.599261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:09.615180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136307e4-34ed-4a42-a2e1-d2d7ede1d394 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.619071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:09.621024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:09.675911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.676733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.677016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.677297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.687062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.690301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:09.706336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:09.707533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:09.708633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:09.712597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 473/953] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:09.714530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 481/954] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:09.723355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:09.847450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14de42c-d0e6-4f03-9b66-e05152cc2a26 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:09.882650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:09.883256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:09.883533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:09.883805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:09.884527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:09.884832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:09.886251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:09.899513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 474/955] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:09.906097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:10.011729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129477cd-f221-44fc-a287-bb36cea5ec46 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:10.032427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:10.032640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:10.052787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:10.052965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:10.053503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.053789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.054072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/938980e29ada4c4eba2a4172845396e1/roles/bf18b261e4e5416b8ded25432ee11a32` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.056486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=938980e29ada4c4eba2a4172845396e1, role_id=bf18b261e4e5416b8ded25432ee11a32)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:10.057949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:10.076336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 482/956] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:53:09 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/938980e29ada4c4eba2a4172845396e1/roles/bf18b261e4e5416b8ded25432ee11a32 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:53:10.167560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0f92b-659f-47fe-9e60-42f81ed35765 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:10.210773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:10.211428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.211709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.211983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.212644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:10.214328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:10.225591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:10.226668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:10.227517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 475/957] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:10.233807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:10.352386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d693051f-9f39-4623-9cb9-3d7c1cdeea52 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:10.392932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:10.393555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.393835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.394112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.394757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:10.395067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:10.396476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf24783-b624-4246-be46-87e1fa98c2a5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:10.402277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 483/958] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:10.409126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:10.414437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed917888-cb96-47d3-8675-e8fb1466afa2 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.414726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed917888-cb96-47d3-8675-e8fb1466afa2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.415008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed917888-cb96-47d3-8675-e8fb1466afa2 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.416680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 484/959] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:10.423464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.423738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.424037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.443307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:10.444525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:10.483807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:10.495317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:10.524416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 485/960] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:10.531208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb11708-3cdf-41a9-b0ab-8134bd0c8907 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:10.570060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:10.570949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.571368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.571782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.572653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:10.573099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:10.574658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:10.580431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 476/961] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:10.589383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:10.701794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782490d-ae95-49d3-81b1-12621e96bfd8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:10.738463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:10.739086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.739331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.739604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.740300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:10.741661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:10.766970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 486/962] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:53:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:10.804639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:10.924534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e14e-6ed0-4afe-827a-136e17f43a91 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:10.969130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:10.969652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:10.969928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:10.970198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:10.970826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:10.972300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:10.974784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING keystone.server.flask.common [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 10:53:10.984218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:10.991409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.sql.core [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 10:53:10.991409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 10:53:10.991409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [parameters: {'user_id': '14f6d7efe01c40f2baa011853caa0011', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 10:53:10.991409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81462) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application result.read() Aug 31 10:53:11.019667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:53:11.020670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application yield resource Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application next(self.gen) Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise self.value Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application session.commit() Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 10:53:11.021668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self.session.flush() Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application compat.raise_( Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 10:53:11.022571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 10:53:11.023484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application conn.query(q) Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application result.read() Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 10:53:11.025812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [parameters: {'user_id': '14f6d7efe01c40f2baa011853caa0011', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:11.027873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 10:53:11.036671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:11.039111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 477/963] 10.222.0.20 () {60 vars in 1131 bytes} [Wed Aug 31 10:53:10 2022] POST /identity/v3/users => generated 160 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:11.041092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:11.069160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:11.150423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e12efa-8eb2-47ce-812c-6800d29b43a1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:11.187172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:11.187805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:11.188108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:11.188386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:11.196037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:11.197495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-74bbeb5c-585a-40cb-b280-fadbb52b867e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:11.210072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:11.211055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 10:53:11.212023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:11.216252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 487/964] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:11.218678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:11.330377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe27bf-8a7d-4041-9515-fa9f32a75efb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:11.366421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:11.367093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:11.367370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:11.367641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:11.368408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:11.368711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:11.370382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371c7b2-d0f4-438a-86ae-0c2b69c5ff75 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:11.415896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 478/965] 10.222.0.20 () {56 vars in 1101 bytes} [Wed Aug 31 10:53:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:11.721597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7c5d-e46c-40bb-8e54-d93436bfead5 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:11.721881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7c5d-e46c-40bb-8e54-d93436bfead5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:11.722165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7c5d-e46c-40bb-8e54-d93436bfead5 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:11.723539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 488/966] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:11.728844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:11.729300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:11.729391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:11.748435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:11.750254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:11.816325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:11.835324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 479/967] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:12.097672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928dc77-d2d4-4827-85c1-09a0f12d9169 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.098167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928dc77-d2d4-4827-85c1-09a0f12d9169 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.098167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928dc77-d2d4-4827-85c1-09a0f12d9169 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.099497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 489/968] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:12.101642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:12.104583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.104867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.105138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.123371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:12.124588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:12.197387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 490/969] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:12.211612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a4ab4d-c368-4f8a-9919-af5617ecaba6 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:12.248276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:12.248878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.249157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.249427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.250089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:12.251450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:12.266689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 480/970] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:53:12 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:12.462918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:12.483038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:12.572366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c8632-af23-41b8-884e-d5cc9c7381e5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:12.614076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:12.614705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.614978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.615247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.625790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:12.630604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:12.631738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:12.632758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 491/971] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 10:53:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:12.637527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:12.684719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:12.742802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f54ec2-7613-4a23-b7df-b96252ff5fdd None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:12.783247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:12.783247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.783247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.783247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.783662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:12.783662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:12.784496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5aa9ed-f3c3-45ca-b1d6-0edda877b6b9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:12.798580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 481/972] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:53:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:12.800057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:12.858233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca6ee8-b297-42f7-94f7-9df3a623557b None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.858557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca6ee8-b297-42f7-94f7-9df3a623557b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.858806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca6ee8-b297-42f7-94f7-9df3a623557b None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.861082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 482/973] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:12.866246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.867917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.868244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.884668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:12.888670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:12.889859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:12.916309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41757cf-cb85-4b3a-8b2b-0048ac2ea649 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:12.958081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:12.958703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:12.959085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:12.959373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:12.961203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 483/974] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:12.976157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:12.977737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:12.992917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:12.993846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:12.994738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:12.994738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:12.994738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:12.996307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 492/975] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:12.999807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.042116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.103517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72087d58-b4c3-4772-8367-a8e1982a5296 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.136686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.137285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.138128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.138403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.139037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:13.139337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.140720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.152094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 484/976] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:13.158616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.237609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.265019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146a1c-b300-44d1-bca0-63afd53c98cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.291980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.302954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.303698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.303982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.304289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.313279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.314731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:13.327297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:13.328570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:13.329562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 493/977] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:13.334142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.342814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a01f68-65b6-48de-a516-cce865f9a907 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.350430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.378401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.379032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.379312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.379583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.380192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.381552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:13.390392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:13.391417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:13.392391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 485/978] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:13.398088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.444602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac388-5b35-4f36-8366-1f89d2c56b1c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.479468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.480096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.480375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.480648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.481925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:13.482228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.483647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.501200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 494/979] 10.222.0.20 () {56 vars in 1149 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:13.506683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.506890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6cdce-b63c-47f1-aaab-7ddb88ff2877 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.544463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.545218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.545504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.546552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.547244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:13.547549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.548994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.554113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 486/980] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:13.559762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.562194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.589257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.621294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47418ebd-cc6a-40cd-b003-6c806c8a2991 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.658994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.659609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.659891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.660186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.670952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f81d0-e12d-4b49-8517-749e36d6bd88 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.711850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.712484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.712763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.713300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.713922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:13.714227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.716299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.722570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 487/981] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:13.728545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.733368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.734820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.790796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.822349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:13.833678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dbb948-6f6c-40c2-8af3-678239130729 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.849980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 495/982] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:53:13.855506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.870683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:13.871288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:13.871568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:13.871875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:13.872686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] RBAC: Adding query filter params (interface=admin, service_id=a7676e80ea4c40c88032e45ac07abeab, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:13.873162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:13.874542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:13.879998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 488/983] 10.222.0.20 () {56 vars in 1249 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/endpoints?service_id=a7676e80ea4c40c88032e45ac07abeab&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:13.885394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:13.970129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a689a6c6-cd33-4e4e-8232-9f0071416cbf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:13.990419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0dc02-2c28-4043-9e27-550ff7fc5001 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.005762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.006390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.006666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.006934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.017247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.018683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.021167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:14.029133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.029933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.030384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.030800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.031566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.033157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:14.033902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:14.035021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:14.036088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:14.036088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:14.036088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:14.036088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 496/984] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:14.040452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:14.042397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 489/985] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:53:13 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 10:53:14.152376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fbfa2-c9bf-4481-b76e-6cdaa4a30788 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.201677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.202332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.202555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.202827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.203507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:14.203817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.205260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acb950-c94e-45a6-be12-11555335ab47 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.219143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 497/986] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:14.225687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:14.246855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:14.258824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa25ff-dac6-4d0c-a93e-fcdc5177480a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.259104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa25ff-dac6-4d0c-a93e-fcdc5177480a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.259384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa25ff-dac6-4d0c-a93e-fcdc5177480a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.260779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 498/987] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:14.266066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.266347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.266620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.285548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:14.286733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:14.340288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a816788-b34e-43a2-81cf-c9e164a53d81 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.342355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:14.361010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 499/988] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:14.377587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.378215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.378490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.378758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.385721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.387148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:14.399310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:14.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:14.401251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 490/989] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:14.409249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:14.514164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b496f298-4052-4f74-ae1d-571593e25c33 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.548734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.549350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.549628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.549898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.550598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:14.550899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.552366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.569102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 500/990] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:53:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:14.570372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:14.645087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:14.651232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:14.676766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6193ec8-b8df-40e0-8752-70e7183edaec None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.703744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:14.713818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.714427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.714707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.714978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.715871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] RBAC: Adding query filter params (role.id=bf18b261e4e5416b8ded25432ee11a32, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=938980e29ada4c4eba2a4172845396e1) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:14.716216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.717643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.723746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 491/991] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:14 2022] GET /identity/v3/role_assignments?user.id=938980e29ada4c4eba2a4172845396e1&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=bf18b261e4e5416b8ded25432ee11a32 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:14.766727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4df70-a29b-4dfb-ab0b-4a6b47be6ef9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.813176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.813819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.814234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.814394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.814963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.816402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:14.826652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:14.828501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:14.830636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 501/992] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:14.833872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:14.951509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba09aaf-e262-4a17-b5f3-ae959a3f9ad8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:14.987460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:14.988158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:14.988440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:14.988712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:14.989337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:14.989669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:14.990973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:14.995895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 492/993] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:15.002062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:15.107151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d926d3b-5cbf-483c-babd-2cbb89a3d36a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:15.117709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:15.142724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:15.143380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:15.143658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:15.143927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:15.144582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:15.144885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:15.146235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:15.157026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 502/994] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:15.158399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:15.168611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:15.266099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d382232-c058-4bf9-93db-dc8748d70522 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:15.299838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:15.300469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:15.300748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:15.301022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:15.301720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] RBAC: Adding query filter params (interface=public, service_id=0edb44e6392848cc8c61df300939f780, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:15.302031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:15.303411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c53745-105b-450f-9612-6299eee718fd None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:15.308584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 493/995] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:53:15 2022] GET /identity/v3/endpoints?service_id=0edb44e6392848cc8c61df300939f780&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:15.314145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:15.417781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d872cd-cfa3-4bd1-b321-0b70da46e648 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:15.451838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b5e03-0680-40af-860e-e6da01add8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:15.452474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b5e03-0680-40af-860e-e6da01add8e6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:15.452755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b5e03-0680-40af-860e-e6da01add8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:15.453032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b5e03-0680-40af-860e-e6da01add8e6 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:15.453637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b5e03-0680-40af-860e-e6da01add8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:15.454999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b5e03-0680-40af-860e-e6da01add8e6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:15.468039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 503/996] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:53:15 2022] GET /identity/v3/services => generated 1727 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:53:16.014456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8233a7-5f85-44fa-8800-c6f914632afc None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.014968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8233a7-5f85-44fa-8800-c6f914632afc None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.015074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8233a7-5f85-44fa-8800-c6f914632afc None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.016568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 494/997] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:16.021514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.021813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.022089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.040674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:16.041779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:16.072213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:16.112925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 504/998] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:16.382620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8233a7-5f85-44fa-8800-c6f914632afc None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:16.456517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:16.490093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8233a7-5f85-44fa-8800-c6f914632afc None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:16.524571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:16.525201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.525478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.525757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.526341 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:16.527703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-db269d89-dc70-4956-92f5-fac2a8eddf77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:16.537017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:16.538040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 10:53:16.538912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:16.543308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 495/999] 10.222.0.20 () {56 vars in 1126 bytes} [Wed Aug 31 10:53:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:16.544717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:16.648091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089e5e08-5512-40f6-a02c-137e0635d168 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:16.676319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29403e-f445-4240-ba5f-32844187cecb None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.676606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29403e-f445-4240-ba5f-32844187cecb None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.676897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29403e-f445-4240-ba5f-32844187cecb None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.678253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 496/1000] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:16.681971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:16.682590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.682867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.683049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.683146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.683326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.683597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.683760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:16.684667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:16.686110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:16.695751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 505/1001] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:16.697003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:16.702812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:16.703984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:16.732265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:16.789725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 497/1002] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:16.806883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89dcb0-da26-4df0-a9f3-3d344ac820cf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:16.842918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:16.843523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.843804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.844109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.844741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:16.845051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:16.846432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:16.851502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 506/1003] 10.222.0.20 () {56 vars in 1122 bytes} [Wed Aug 31 10:53:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:16.857005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:16.921847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:16.961245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d023-fa97-4743-aad3-16dce4347f99 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:16.996036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:16.996692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:16.996974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:16.997380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:16.998059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:16.999448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:17.017594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 498/1004] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:53:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:53:17.058248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:17.161893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:17.171030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb77f76-3ed3-4f57-b432-350e1cf38c6b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:17.213243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:17.213789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:17.214132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:17.214400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] PATH_INFO: `/v3/services/image` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:17.214982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:17.216911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:17.225945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:17.226933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:17.227787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 507/1005] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 31 10:53:17 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:17.233544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:17.336768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4185e-9a12-419f-838b-60cd91d82aa2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:17.360654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:17.373993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:17.374601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:17.374880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:17.375153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:17.375786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] RBAC: Adding query filter params (name=image) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:17.376118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:17.377474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee5669-923b-403c-b250-b55aedb1b299 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:17.386850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 499/1006] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:17.388610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:17.410445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38070afe-f791-428b-adfa-7b49058daf33 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:17.410754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38070afe-f791-428b-adfa-7b49058daf33 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:17.411000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38070afe-f791-428b-adfa-7b49058daf33 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:17.412411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 500/1007] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:17.417778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:17.418057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:17.418330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:17.437870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:17.439070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:17.502723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f80cc84-9f4a-4a06-a409-c67b1bd9d624 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:17.512962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 501/1008] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:17 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:17.541782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:17.542401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:17.542665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:17.542936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:17.544094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] RBAC: Adding query filter params (type=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:17.544403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:17.545763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:17.559972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 508/1009] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:17 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:17.789654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:17.893318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6c5b44-fb78-4496-9e88-009951927ea2 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:17.911197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:17.929617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:17.930250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:17.930528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:17.930799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:17.931386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:17.932847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:17.942259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:17.943267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 10:53:17.944737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:17.948099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 502/1010] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:17.949401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:17.974256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:18.059413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7c9d6-8a83-4273-9550-b832de6e82e8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:18.094385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:18.094994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:18.095274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:18.095546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:18.096209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:18.096516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:18.097964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:18.108225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 509/1011] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:18.109890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:18.223630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c4642f-21f8-4f09-a675-a22c44e4957b None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:18.259417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:18.260047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:18.260338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:18.260614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:18.261327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:18.261641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:18.263119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdde996-7da9-4361-a886-37fc630605d9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:18.272819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 503/1012] 10.222.0.20 () {56 vars in 1112 bytes} [Wed Aug 31 10:53:18 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:18.273820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:18.371282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:18.389040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265aa1e-9d8f-453b-a2b6-94432f7cd150 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:18.423795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ba3a4-2b43-47be-851e-a86e4d50bfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:18.424448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ba3a4-2b43-47be-851e-a86e4d50bfa9 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:18.424730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ba3a4-2b43-47be-851e-a86e4d50bfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:18.425004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ba3a4-2b43-47be-851e-a86e4d50bfa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:18.425666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ba3a4-2b43-47be-851e-a86e4d50bfa9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:18.427031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94ba3a4-2b43-47be-851e-a86e4d50bfa9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:18.456036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 510/1013] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:53:18 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:18.963275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46701f-7f24-4986-a698-2733bfd8870a None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:18.963600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46701f-7f24-4986-a698-2733bfd8870a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:18.963822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46701f-7f24-4986-a698-2733bfd8870a None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:18.965457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 504/1014] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:18.970773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:18.971265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:18.971343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:18.990424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:18.991642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:19.070024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 511/1015] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:18 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:19.335370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46701f-7f24-4986-a698-2733bfd8870a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:19.384764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:19.440957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46701f-7f24-4986-a698-2733bfd8870a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:19.474608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fb87e-ef9f-43f6-9d68-c9b9092d2340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:19.475230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fb87e-ef9f-43f6-9d68-c9b9092d2340 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:19.475504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fb87e-ef9f-43f6-9d68-c9b9092d2340 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:19.475772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fb87e-ef9f-43f6-9d68-c9b9092d2340 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:19.476903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fb87e-ef9f-43f6-9d68-c9b9092d2340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:19.478319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9fb87e-ef9f-43f6-9d68-c9b9092d2340 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:19.480201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 505/1016] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:19.486284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:19.491976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feabf91-d1d6-4e61-a637-d62d49361186 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:19.491976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feabf91-d1d6-4e61-a637-d62d49361186 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:19.492214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feabf91-d1d6-4e61-a637-d62d49361186 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:19.493372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 506/1017] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:19.498572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:19.498857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:19.499132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:19.518722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:19.519898 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:19.557416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:19.598111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 507/1018] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:19 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:19.601035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414534a-6f46-4797-9977-b1e86473dac5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:19.638293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a5632-4805-4025-acba-18e0d1d9e015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:19.638906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a5632-4805-4025-acba-18e0d1d9e015 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:19.639191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a5632-4805-4025-acba-18e0d1d9e015 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:19.639469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a5632-4805-4025-acba-18e0d1d9e015 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:19.640170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a5632-4805-4025-acba-18e0d1d9e015 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:19.641618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a5632-4805-4025-acba-18e0d1d9e015 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:19.680042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 512/1019] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:53:19 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:19.870247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:19.917111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:19.975665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e9408-4192-4b50-94d6-3f839d5de7bc None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:20.010367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b61f2-6e6f-4ce2-8e67-1ecb49500766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:20.010972 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b61f2-6e6f-4ce2-8e67-1ecb49500766 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:20.011247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b61f2-6e6f-4ce2-8e67-1ecb49500766 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:20.011514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b61f2-6e6f-4ce2-8e67-1ecb49500766 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:20.012780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b61f2-6e6f-4ce2-8e67-1ecb49500766 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:20.014169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b61f2-6e6f-4ce2-8e67-1ecb49500766 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:20.029863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 508/1020] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:53:19 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:20.518345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88a3e-07ba-4c6a-856e-982e5cad604e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:20.518880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88a3e-07ba-4c6a-856e-982e5cad604e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:20.518880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff88a3e-07ba-4c6a-856e-982e5cad604e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:20.520270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 513/1021] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:20.525348 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:20.525780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:20.525886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:20.544630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:20.545823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:20.620979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 509/1022] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:20 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:20.891253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff88a3e-07ba-4c6a-856e-982e5cad604e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:20.996885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff88a3e-07ba-4c6a-856e-982e5cad604e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:21.015668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:21.036422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:21.037174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.037303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.037521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.049703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:21.054702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:21.055692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:21.061570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 514/1023] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 10:53:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:21.066752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:21.099157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:21.177647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ed9c7-cc75-4a15-a319-5e7415a4fe40 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:21.212448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:21.213314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.213589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.213861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.214527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:21.214830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:21.216289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:21.228139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 510/1024] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:53:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:21.231523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:21.335656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee03d3d-48f4-4a9f-8464-98929720cb45 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:21.370469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:21.371098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.371928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.372233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.382776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:21.384288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:21.400857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:21.401907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:21.408791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:21.408791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:21.408791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:21.408791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 515/1025] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:21.409027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:21.514865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:21.525339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2421f157-6be3-4675-9e36-5b1998a6a6fc None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:21.572960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:21.573580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.573858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.574130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.574813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:21.575114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:21.576515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa43379-840c-4da2-8c5d-205ffb2f6a08 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:21.589607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 511/1026] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:21.595629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:21.627641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:21.642534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76c83-132d-44c8-bd24-81da990c5429 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.642847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76c83-132d-44c8-bd24-81da990c5429 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.643089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd76c83-132d-44c8-bd24-81da990c5429 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.644656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 512/1027] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:21.649474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.649756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.650024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.670078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:21.671370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:21.702791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc99fc-df0a-47a7-80f9-af7af02ff141 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:21.739931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:21.740598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.740874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.741145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.741678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 513/1028] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:21.749486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:21.751172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:21.762814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:21.763783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:53:21.764769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:21.765308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 516/1029] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:53:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:21.771265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:21.857348 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:21.877316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a203c-d3a3-4709-9e0c-e451eb3779d5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:21.912547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:21.913163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:21.913443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:21.913716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:21.914454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:21.914760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:21.916219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2293eae9-3dfb-43e7-9362-fe6a617c517c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:21.940055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 514/1030] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:53:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:21.941444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.015196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b953-d810-4076-bd11-97ea14d6e5e9 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.016088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b953-d810-4076-bd11-97ea14d6e5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.016372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90b953-d810-4076-bd11-97ea14d6e5e9 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.017896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 515/1031] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:22.019969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90b953-d810-4076-bd11-97ea14d6e5e9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.042275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.053540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1171afe-16e7-430c-a73f-e4b0101a48d9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.088693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.089299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.089574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.089843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.125781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90b953-d810-4076-bd11-97ea14d6e5e9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.159191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.160768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.160893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ba642-0926-40a1-9baf-3b24abbb89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.161514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba642-0926-40a1-9baf-3b24abbb89fc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.161795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba642-0926-40a1-9baf-3b24abbb89fc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.162066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ba642-0926-40a1-9baf-3b24abbb89fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.163183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ba642-0926-40a1-9baf-3b24abbb89fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.164630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ba642-0926-40a1-9baf-3b24abbb89fc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.166249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 516/1032] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:22.167374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.167657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.167930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.185865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:22.187050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:22.189131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.252342 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.258289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 517/1033] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:22.267867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 517/1034] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:22.269050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.269182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.376386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfc7de-1790-4d17-9aff-e8c7d4b116ea None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.381262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4361d04-bf08-4459-b08f-e0972d19eb69 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.410957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.411593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.411873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.412177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.421238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.422123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.422495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.422769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.423227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.423939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.424383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.426169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.438196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:22.439937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:22.441112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:22.442199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:22.442199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:22.442199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:22.442199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 518/1035] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:22.447343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.457902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 518/1036] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:53:22 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:22.509449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.529070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.578216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c959b5-2603-4818-96fe-49fbb853171e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.618412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.619068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.619353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.619548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.620229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:22.620531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.622807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.634846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 519/1037] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:22.640760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.660800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d5d77-4fcb-41b4-ac5d-009f8c5a3b40 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.703014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.703651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.704148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.704334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] PATH_INFO: `/v3/services/image` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.705035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.706410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:22.717584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:22.718641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:22.719484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 519/1038] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/services/image => generated 86 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:22.725018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.752623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12c3cf-8f2d-4731-9d1b-37a733fd5cca None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.754302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.795195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.795940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.796337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.796677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.797903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:22.804638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.806100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:22.818797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:22.819776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:22.820712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 520/1039] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:22.827940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.846703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab19a9b-8980-4b5e-a9c7-fdd0fdd60035 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.889097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.889631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.889932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.890200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.890845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:22.891146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.892578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:22.898228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 520/1040] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:22.905117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:22.948716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27b28c-0b02-4504-afca-fd2eaf70b887 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:22.992699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:22.993481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:22.993769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:22.994046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:22.994745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:22.995055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:22.996542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:23.010444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 521/1041] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:23.010872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc5f23-6ed9-4707-aeea-eaf3ade152f4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:23.016923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:23.039281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:23.044238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:23.053921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:23.054528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:23.054806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:23.055079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:23.055709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] RBAC: Adding query filter params (type=image) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:23.056044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:23.057407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:23.063291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 521/1042] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:23.069068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:23.128540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78ce9bb-389a-414d-92b8-45defdf9bbd2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:23.166623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:23.167228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:23.167502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:23.167772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:23.169075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] RBAC: Adding query filter params (role.id=bf18b261e4e5416b8ded25432ee11a32, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=508f557ac5e34c67a31e4d55f8142a0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:23.169382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:23.170887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:23.176729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 522/1043] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:23 2022] GET /identity/v3/role_assignments?user.id=508f557ac5e34c67a31e4d55f8142a0f&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=bf18b261e4e5416b8ded25432ee11a32 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:23.182112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272ef24-66d8-43d3-9e43-c955da33bd95 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:23.222958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:23.223640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:23.223924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:23.224236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:23.224982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] RBAC: Adding query filter params (interface=public, service_id=8d317c00230b46eeb43943e135a4c94c, region_id=RegionOne) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:23.225293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:23.226728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3909d1-cf81-47f8-815d-e357068da874 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:23.238696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 522/1044] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:53:23 2022] GET /identity/v3/endpoints?service_id=8d317c00230b46eeb43943e135a4c94c&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:23.239589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:23.334808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:23.352057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9082a7-31cf-4c12-8786-bb0c5f295a70 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:23.387520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de95c94f-6329-43e5-af03-16d36a8cffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:23.388245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c94f-6329-43e5-af03-16d36a8cffb7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:23.388529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c94f-6329-43e5-af03-16d36a8cffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:23.388802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95c94f-6329-43e5-af03-16d36a8cffb7 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:23.389408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95c94f-6329-43e5-af03-16d36a8cffb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:23.390780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de95c94f-6329-43e5-af03-16d36a8cffb7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:23.404053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 523/1045] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:53:23 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 10:53:24.502840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb899369-c918-44f6-a7d9-cd0ed5df5c5e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:24.503389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb899369-c918-44f6-a7d9-cd0ed5df5c5e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:24.503389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb899369-c918-44f6-a7d9-cd0ed5df5c5e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:24.504922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 523/1046] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:24.510069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29780414-e060-455f-a53f-9e39a30f917f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:24.510459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29780414-e060-455f-a53f-9e39a30f917f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:24.510656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29780414-e060-455f-a53f-9e39a30f917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:24.529202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-29780414-e060-455f-a53f-9e39a30f917f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:24.530396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-29780414-e060-455f-a53f-9e39a30f917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:24.601943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 524/1047] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:24.876676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb899369-c918-44f6-a7d9-cd0ed5df5c5e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:24.933733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:24.989040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb899369-c918-44f6-a7d9-cd0ed5df5c5e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.024497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3df1d0-b363-467d-9d61-4de563639124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.025135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3df1d0-b363-467d-9d61-4de563639124 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.025417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3df1d0-b363-467d-9d61-4de563639124 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.025693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3df1d0-b363-467d-9d61-4de563639124 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.037159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3df1d0-b363-467d-9d61-4de563639124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-da3df1d0-b363-467d-9d61-4de563639124 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:25.041946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:25.042947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:25.043917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:25.048191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 524/1048] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:53:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:25.049475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29780414-e060-455f-a53f-9e39a30f917f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.107325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a62df-f154-419c-ba96-fadff69f271d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.107670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a62df-f154-419c-ba96-fadff69f271d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.107868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a62df-f154-419c-ba96-fadff69f271d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.109252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 525/1049] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:25.114304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.114593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.114870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.132883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:25.134074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:25.139588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.165400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29780414-e060-455f-a53f-9e39a30f917f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.205939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.206547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.207018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.207161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 526/1050] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:25.207265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.207746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:25.208088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:25.209448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db64206-108b-4c1b-8672-f20044e5bea6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:25.219264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 525/1051] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:25.225000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.268895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.329055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78f13a-1337-4050-85c1-12d92c42fbf7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.337050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33bcf-9a7a-4b8b-b77f-9d49669d3e17 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.337334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33bcf-9a7a-4b8b-b77f-9d49669d3e17 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.337876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c33bcf-9a7a-4b8b-b77f-9d49669d3e17 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.339222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 526/1052] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:25.344787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.345069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.345340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.364216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:25.365312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.365945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.366066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:25.366221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.366485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] PATH_INFO: `/v3/users/54c3547b69c84876840623f0809fb6cc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.381058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] RBAC: Authorizing `identity:get_user(user_id=54c3547b69c84876840623f0809fb6cc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:25.382485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:25.384459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 527/1053] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/users/54c3547b69c84876840623f0809fb6cc => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:25.390707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.446558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 527/1054] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:25.483449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.492726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.497487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7aad3-7984-4403-ade8-d53700830aca None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.536548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.537217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.537459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.537821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.559838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.592198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dabaaa-72bc-4992-8b59-5afc5197a8ed None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.618464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:25.620366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:25.626544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.627172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.627450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.627719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.639485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:25.644399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:25.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:25.646228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:25.650611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 528/1055] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:25.651340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.731977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.741476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 528/1056] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:53:25.741476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.758563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144ab2f-bb8a-48c3-be9c-116b37c64457 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.791573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.798463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.799072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.799351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.799621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.800308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:25.800608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:25.801969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:25.810784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 529/1057] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:25.815643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.836768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c561057-ab7d-49a3-85e9-39c619e214d8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.879993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.880572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.880846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.881122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.881691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:25.883035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:25.891815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:25.893076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:25.894017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 529/1058] 10.222.0.20 () {56 vars in 1105 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:25.894859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.916749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e21dbb-7148-4909-9572-85fce739e764 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:25.951792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:25.952460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:25.952742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:25.953024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:25.954205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] RBAC: Authorizing `identity:get_project(project_id=43945b30ba42482cad079a3159431865)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:25.954561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:25.955818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:25.957582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 530/1059] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/projects/43945b30ba42482cad079a3159431865 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:25.958352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:25.998190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b27a565-9bab-4581-bf08-10f8c0ef41a5 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.019500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.032267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.032896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.033175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.033850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.046409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.047850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.058022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21aad0-b00a-4683-b264-9f097c2f79df None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:26.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:26.063452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:26.064414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:26.064414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:26.064414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:26.064414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 530/1060] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:26.064675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.096078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.096681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.096958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.097232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.097848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:26.098149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.099501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.104259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 531/1061] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:26.105031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.178826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a54f2f-88f0-47d7-a466-cd917141d36c None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.192290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.209225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7bbae-b7b5-4b0e-b4fc-9becfce360a6 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.211924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.212811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.213089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.213360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.214074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=43945b30ba42482cad079a3159431865, user.id=54c3547b69c84876840623f0809fb6cc) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:26.214378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.216340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.221896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 531/1062] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/role_assignments?user.id=54c3547b69c84876840623f0809fb6cc&scope.project.id=43945b30ba42482cad079a3159431865&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:26.222527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.237921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.249628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.250232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.250512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.250781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.251465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:26.251736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.253206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.267272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 532/1063] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:26.273118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.339087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b1388-b593-43a6-adf2-f674a2c51999 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.376100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.376721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.377002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.377279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.377911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] RBAC: Adding query filter params (type=image) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:26.378491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.379585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.387349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 532/1064] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:26.390232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a80484-ea67-4662-b451-bfdcaf083d94 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.392661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.427982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.428647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.428979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.429198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.436933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.438384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.439313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:26.454937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:26.456218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:53:26.457270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:26.460981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 533/1065] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:26.461846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.504510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bab15-e532-471e-bc6c-a526fdba53cf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.512457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.544147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.544831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.545043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.545282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.545950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.547311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.567866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 533/1066] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:53:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:53:26.583593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2348257-83e5-415d-809c-b593b2dd88c9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.630044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.630675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.630957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.631224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.632000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:26.632346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.633890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.661005 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 534/1067] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:26.662712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.780474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.788031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59c3c8f-f5ce-4a5f-9e0e-3ef9d4aa33ab None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:26.825033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:26.825667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:26.825943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:26.826213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:26.838502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:26.840487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:26.856950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:26.857965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:26.858958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:26.858958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:26.858958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:26.858958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 534/1068] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:26.864471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:26.973352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:26.976997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b78705-3365-46fc-97b4-2d377b1477d4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:27.014017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:27.014628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:27.014906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:27.015175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:27.015825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:27.016156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:27.017538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:27.035111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 535/1069] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:27.036399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:27.140688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda5afe0-fabe-4cb9-876a-484f49162be2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:27.175671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:27.176325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:27.176600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:27.176868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:27.184080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:27.185530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:27.198043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:27.199030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:27.201332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 535/1070] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:27.206028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:27.312267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a657d-7e64-4b00-9a08-eff170d153ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:27.349757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:27.350377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:27.350659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:27.350936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:27.351647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:27.351954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:27.353427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5defd809-85c1-4513-832e-84b7b0350ea7 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:27.366778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 536/1071] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:27.372791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:27.387142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:27.478998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ef5ca-2323-444c-8ccb-76532bad1459 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:27.515677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8887a-b1bc-4ff3-aab8-a3df1c2469a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:27.516386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8887a-b1bc-4ff3-aab8-a3df1c2469a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:27.516668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8887a-b1bc-4ff3-aab8-a3df1c2469a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:27.516943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8887a-b1bc-4ff3-aab8-a3df1c2469a8 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/508f557ac5e34c67a31e4d55f8142a0f/roles/bf18b261e4e5416b8ded25432ee11a32` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:27.519132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8887a-b1bc-4ff3-aab8-a3df1c2469a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=508f557ac5e34c67a31e4d55f8142a0f, role_id=bf18b261e4e5416b8ded25432ee11a32)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:27.520626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8887a-b1bc-4ff3-aab8-a3df1c2469a8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:27.537545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 536/1072] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:53:27 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/508f557ac5e34c67a31e4d55f8142a0f/roles/bf18b261e4e5416b8ded25432ee11a32 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:53:28.166102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb134c2-7948-469d-951f-6f118fc95e04 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:28.166767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb134c2-7948-469d-951f-6f118fc95e04 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:28.166767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb134c2-7948-469d-951f-6f118fc95e04 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:28.168927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 537/1073] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:28.174317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8cf7-29a4-4418-b921-d043b20e0815 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:28.174860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8cf7-29a4-4418-b921-d043b20e0815 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:28.174860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad8cf7-29a4-4418-b921-d043b20e0815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:28.198019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-92ad8cf7-29a4-4418-b921-d043b20e0815 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:28.199762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-92ad8cf7-29a4-4418-b921-d043b20e0815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:28.278010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:28.290696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 537/1074] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:28.558965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb134c2-7948-469d-951f-6f118fc95e04 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:28.581151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:28.610698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c2785-d28b-4753-8840-5ce7acbcd144 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:28.611029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c2785-d28b-4753-8840-5ce7acbcd144 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:28.611263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c2785-d28b-4753-8840-5ce7acbcd144 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:28.612685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 538/1075] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:28.618138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:28.618417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:28.618769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:28.638375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:28.639727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:28.676901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb134c2-7948-469d-951f-6f118fc95e04 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:28.714042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 539/1076] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:28.715694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:28.716362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:28.716720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:28.717043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:28.728180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:28.733494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:28.734423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:28.736834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 538/1077] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:53:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:28.740864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:28.851875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e714-14ad-4ce2-a45b-0251fc65726e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:28.881532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:28.888310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:28.888945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:28.889231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:28.889508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:28.890179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:28.890484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:28.891853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:28.900535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 540/1078] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:28.906195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:28.988232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:28.990622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:29.012952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a55b72-41f9-4763-a952-eee455b9d1a2 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:29.054081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:29.054711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.054985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.055254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] PATH_INFO: `/v3/users/54c3547b69c84876840623f0809fb6cc` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.056394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] RBAC: Authorizing `identity:get_user(user_id=54c3547b69c84876840623f0809fb6cc)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:29.057851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:29.060473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 539/1079] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:28 2022] GET /identity/v3/users/54c3547b69c84876840623f0809fb6cc => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:29.065856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:29.105247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc34de-6051-4907-8a92-605232ab32c2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:29.147842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:29.148553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.148835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.149108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.176106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bce7b-d324-4ddc-bcbb-1872c87ecc1a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:29.210319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:29.210871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:29.211251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.211527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.211799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.213353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] RBAC: Authorizing `identity:get_project(project_id=43945b30ba42482cad079a3159431865)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:29.215390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:29.217258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 540/1080] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:29 2022] GET /identity/v3/projects/43945b30ba42482cad079a3159431865 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:29.222983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:29.228278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:29.229752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:29.241226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:29.333833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb316cb-0436-429d-aa31-204ae6d0f412 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:29.334134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 541/1081] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:53:29.340232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:29.373119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9ef8b-c4ff-4971-b941-ab3da429738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:29.374028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ef8b-c4ff-4971-b941-ab3da429738d None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.374305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ef8b-c4ff-4971-b941-ab3da429738d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.374580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9ef8b-c4ff-4971-b941-ab3da429738d None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865/users/54c3547b69c84876840623f0809fb6cc/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.377221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9ef8b-c4ff-4971-b941-ab3da429738d None admin] RBAC: Authorizing `identity:create_grant(project_id=43945b30ba42482cad079a3159431865, user_id=54c3547b69c84876840623f0809fb6cc, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:29.378655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9ef8b-c4ff-4971-b941-ab3da429738d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:29.393245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 541/1082] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:53:29 2022] PUT /identity/v3/projects/43945b30ba42482cad079a3159431865/users/54c3547b69c84876840623f0809fb6cc/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 10:53:29.441740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:29.454735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f2f6d-01ab-4d8d-a100-5ab24d2487f9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:29.494097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:29.495132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.495606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.496172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.501884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc028-91c5-45d7-ac35-6e2d3ba11a93 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.502199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc028-91c5-45d7-ac35-6e2d3ba11a93 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.502442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bc028-91c5-45d7-ac35-6e2d3ba11a93 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.503840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 542/1083] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:29.509455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:29.509901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.510186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.510459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.510916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:29.529654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:29.530695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:29.532430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:29.532430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:29.532430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:29.532430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 542/1084] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:53:29.532681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:29.533611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:29.538007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:29.616324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 543/1085] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:29.657590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a87df-aebd-493c-82a8-8272373757cc None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:29.694459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:29.695075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:29.695356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:29.695628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:29.696310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:29.696617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:29.697984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:29.716056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 543/1086] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:29.906499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:29.914947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:30.019870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b6ead9-ae7a-4157-a87c-bf3e63fd0809 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:30.061085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:30.061719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:30.061993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:30.062262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:30.072441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:30.077253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:30.078290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:30.079683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 544/1087] 10.222.0.20 () {56 vars in 1120 bytes} [Wed Aug 31 10:53:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:30.084285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:30.088397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:30.188650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092171f9-59de-4312-b302-24f1b859f044 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:30.235082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:30.235943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:30.235943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:30.236235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:30.236901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:30.237206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:30.238562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:30.249132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 544/1088] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:53:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:30.256723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:30.330176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:30.365280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511c851-6620-497c-bdc7-c35f2401eca8 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:30.399883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:30.400531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:30.400804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:30.401069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:30.412094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:30.413505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:30.428067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:30.429253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:30.430155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:30.430155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:30.430155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:30.430155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 545/1089] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:30.435486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:30.506296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:30.545499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03c282-1060-434c-9e1a-b05a7a03c24e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:30.579494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:30.580136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:30.580420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:30.580693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:30.581359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:30.581663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:30.583024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:30.601712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 545/1090] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:30.602879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:30.700549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0588a-e82e-4618-a06d-af60297e7452 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:30.718599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:30.736203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:30.736830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:30.737111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:30.737386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:30.744794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:30.746281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:53:30.757233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:30.758169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:30.759260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 546/1091] 10.222.0.20 () {56 vars in 1093 bytes} [Wed Aug 31 10:53:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:30.764265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:30.889458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc46e-755f-437f-92b5-e0f438ea6361 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:30.915449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:30.925026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:30.925632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:30.925911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:30.926184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:30.926912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:30.927217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:30.928667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:30.945816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 546/1092] 10.222.0.20 () {56 vars in 1133 bytes} [Wed Aug 31 10:53:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:30.952100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.050793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc54f-cc60-4d7b-a148-1255eede996e None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:31.087458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72576-a312-4457-b9f5-ac48b8b5dcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:31.088084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72576-a312-4457-b9f5-ac48b8b5dcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.088367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72576-a312-4457-b9f5-ac48b8b5dcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.088640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72576-a312-4457-b9f5-ac48b8b5dcc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.137203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:31.160208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de72576-a312-4457-b9f5-ac48b8b5dcc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:31.161661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de72576-a312-4457-b9f5-ac48b8b5dcc8 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:31.267777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 547/1093] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:31.273781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.380672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb00df-3122-4c59-aec3-be4a020fb96d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:31.384417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c99345-2b63-425a-95e7-cb8b57d995e5 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.384771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c99345-2b63-425a-95e7-cb8b57d995e5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.385112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c99345-2b63-425a-95e7-cb8b57d995e5 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.386554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 548/1094] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:53:31.391562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.391846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.392156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.410248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:31.411441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:31.418536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:31.419162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.419432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.419710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.430229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:31.431699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:53:31.446715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:31.447838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:53:31.453766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:53:31.453766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:53:31.453766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:31.453766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 547/1095] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:31.453766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.481640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 549/1096] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:31.489486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:31.553480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ed5a9-7d1a-4870-a117-54eea1b2d57a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:31.589286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:31.590401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.590854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.591127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.591761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:31.592097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:31.593453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:31.605187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 548/1097] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:31.611657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.622823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:31.715077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f162785-6572-4b3c-a199-5336627b7301 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:31.751098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:31.751729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.755379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.755379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.755693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.760702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:31.762400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:53:31.776603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:31.778340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:31.779939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 550/1098] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:31.781275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.861925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:31.864250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:31.870416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd327d-aa24-4e35-997d-9f12c2c15612 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:31.897868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b75014-3c62-473c-a8e4-06d9850c33fa None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:31.908198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:31.908841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.909114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.909381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.920574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:31.925498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:31.926614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:31.927668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 549/1099] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:31.932688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:31.938487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:31.939187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:31.939464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:31.939732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:31.940461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:31.940767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:31.942270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:31.954571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 551/1100] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:31.961295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:32.040924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4ea19-730c-4733-a1e7-53571969d840 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:32.073328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06940ec-8873-4ed7-9243-35d3550659f9 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:32.081499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:32.082113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:32.082450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:32.082731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:32.083400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:32.083702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:32.085114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:32.094699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 550/1101] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:32.100399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:32.102019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:32.111089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:32.111686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:32.111961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:32.112259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:32.112977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] RBAC: Adding query filter params (role.id=bf18b261e4e5416b8ded25432ee11a32, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=508f557ac5e34c67a31e4d55f8142a0f) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:32.113284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:32.114702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:32.119798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:32.132062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 552/1102] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:31 2022] GET /identity/v3/role_assignments?user.id=508f557ac5e34c67a31e4d55f8142a0f&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=bf18b261e4e5416b8ded25432ee11a32 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:53:32.214353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eabf734-eda9-4779-84df-963dd62fee70 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:32.254147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:32.254807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:32.255095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:32.255370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] PATH_INFO: `/v3/users/54c3547b69c84876840623f0809fb6cc` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:32.256649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] RBAC: Authorizing `identity:get_user(user_id=54c3547b69c84876840623f0809fb6cc)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:32.258138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:32.259868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 551/1103] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:32 2022] GET /identity/v3/users/54c3547b69c84876840623f0809fb6cc => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:32.266201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:32.381470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752a80b-3342-43aa-8e81-ef0bb0e6d6ba None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:32.416425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:32.417032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:32.417305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:32.418148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:32.482629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:32.492929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:32.494387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:32.599339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 553/1104] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:32.605870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:32.699265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:32.715883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2054dcb-4d64-46e7-81e2-7b708d920d54 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:32.753081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316bb37-b2af-4e57-85a0-b98c4d8a6684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:32.753714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316bb37-b2af-4e57-85a0-b98c4d8a6684 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:32.753998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316bb37-b2af-4e57-85a0-b98c4d8a6684 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:32.754270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316bb37-b2af-4e57-85a0-b98c4d8a6684 None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:32.755412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316bb37-b2af-4e57-85a0-b98c4d8a6684 None admin] RBAC: Authorizing `identity:get_project(project_id=43945b30ba42482cad079a3159431865)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:32.756887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316bb37-b2af-4e57-85a0-b98c4d8a6684 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:32.758844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 552/1105] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:32 2022] GET /identity/v3/projects/43945b30ba42482cad079a3159431865 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:32.765009 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:32.866335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:32.876269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b15146-890c-4c6c-a7e4-0ebc9af9c9dd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:32.912871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:32.913491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:32.913771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:32.914048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:32.914794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=43945b30ba42482cad079a3159431865, user.id=54c3547b69c84876840623f0809fb6cc) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:32.915102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:32.916623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f7eae-aafc-4754-a776-323f589c2f02 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:32.923803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 554/1106] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:32 2022] GET /identity/v3/role_assignments?user.id=54c3547b69c84876840623f0809fb6cc&scope.project.id=43945b30ba42482cad079a3159431865&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:34.891331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c6f5d-c8cf-4488-8b00-3e4bb8e23afa None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:34.891970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c6f5d-c8cf-4488-8b00-3e4bb8e23afa None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:34.891970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c6f5d-c8cf-4488-8b00-3e4bb8e23afa None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:34.893325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 553/1107] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:34.898619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:34.899098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:34.899200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:34.918075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:34.919248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:34.997750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 555/1108] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:35.285010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c6f5d-c8cf-4488-8b00-3e4bb8e23afa None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:35.386084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c6f5d-c8cf-4488-8b00-3e4bb8e23afa None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:35.420061 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786533c-7f36-447f-85e8-063f89aab0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:35.420687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786533c-7f36-447f-85e8-063f89aab0d6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:35.420965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786533c-7f36-447f-85e8-063f89aab0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:35.421237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786533c-7f36-447f-85e8-063f89aab0d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:35.422346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786533c-7f36-447f-85e8-063f89aab0d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:35.423770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d786533c-7f36-447f-85e8-063f89aab0d6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:35.425379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 554/1109] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:35.431244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:35.514350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:35.528920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140707c1-a057-4d6a-a296-c26aac194746 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:35.562679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add6057-f14a-46f0-909f-5fae9f5da849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:35.563212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add6057-f14a-46f0-909f-5fae9f5da849 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:35.563489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add6057-f14a-46f0-909f-5fae9f5da849 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:35.563758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add6057-f14a-46f0-909f-5fae9f5da849 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:35.564443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add6057-f14a-46f0-909f-5fae9f5da849 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:35.565867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add6057-f14a-46f0-909f-5fae9f5da849 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:35.587485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 556/1110] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:53:35 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:37.563210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c127a-9644-4961-a58f-d674d154a4c3 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:37.563766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c127a-9644-4961-a58f-d674d154a4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:37.563766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c127a-9644-4961-a58f-d674d154a4c3 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:37.565155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 555/1111] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:37.570320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:37.570728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:37.570863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:37.589007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:37.590794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:37.656513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 557/1112] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:37.932759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c127a-9644-4961-a58f-d674d154a4c3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:37.958316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:38.038514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c127a-9644-4961-a58f-d674d154a4c3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:38.072840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:38.073492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:38.073805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:38.074084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] PATH_INFO: `/v3/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:38.075485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] RBAC: Authorizing `identity:get_role(role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:38.076877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:38.078227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 556/1113] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:37 2022] GET /identity/v3/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:38.084102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:38.155339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:38.185880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab41e2-a7b7-446a-a9a6-ab2516bfd9ae None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:38.219596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:38.220253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:38.220536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:38.220811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] PATH_INFO: `/v3/users/acf02b4c282b4f62853708452fa649e0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:38.234594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] RBAC: Authorizing `identity:get_user(user_id=acf02b4c282b4f62853708452fa649e0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:38.236070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:38.237723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 558/1114] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:38 2022] GET /identity/v3/users/acf02b4c282b4f62853708452fa649e0 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:38.243644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:38.341192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:38.351134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e2882-38cf-4b36-8c65-737c7ef16fff None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:38.386779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:38.387379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:38.387662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:38.387937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:38.456471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:38.457821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:38.547310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:38.557949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 557/1115] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:38.564770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:38.670849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd377534-d039-4035-8164-b5c318ce326f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:38.698449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:38.704831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b36d06-0515-4925-9207-c93243f87af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:38.705466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b36d06-0515-4925-9207-c93243f87af2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:38.705744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b36d06-0515-4925-9207-c93243f87af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:38.706017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b36d06-0515-4925-9207-c93243f87af2 None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:38.707194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b36d06-0515-4925-9207-c93243f87af2 None admin] RBAC: Authorizing `identity:get_project(project_id=43945b30ba42482cad079a3159431865)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:38.708653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b36d06-0515-4925-9207-c93243f87af2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:38.710306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 559/1116] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:38 2022] GET /identity/v3/projects/43945b30ba42482cad079a3159431865 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:38.716767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:38.814546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c8d5d-963e-4784-ba0f-71bdc2906fd0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:38.848521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:38.849126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:38.849404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:38.849676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:38.850408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] RBAC: Adding query filter params (role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=43945b30ba42482cad079a3159431865, user.id=acf02b4c282b4f62853708452fa649e0) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:38.850714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:38.852156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76b623-8825-4cee-83e4-c49124c18197 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:38.858241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 558/1117] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:38 2022] GET /identity/v3/role_assignments?user.id=acf02b4c282b4f62853708452fa649e0&scope.project.id=43945b30ba42482cad079a3159431865&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:40.758573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76969-c270-449e-a63c-72a03945e9b6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:40.759144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76969-c270-449e-a63c-72a03945e9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:40.759144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76969-c270-449e-a63c-72a03945e9b6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:40.760487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 560/1118] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:40.765861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:40.766331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:40.766430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:40.785013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:40.786210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:40.836019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:40.854095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 559/1119] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:41.123165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76969-c270-449e-a63c-72a03945e9b6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:41.224336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76969-c270-449e-a63c-72a03945e9b6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:41.257787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:41.258417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:41.258690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:41.258959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] PATH_INFO: `/v3/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:41.260438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] RBAC: Authorizing `identity:get_role(role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:41.262383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:41.264000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 561/1120] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:41 2022] GET /identity/v3/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:41.269855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:41.368091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022c13f1-4a15-4508-a441-be112cc2876f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:41.402463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:41.403006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:41.403284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:41.403555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] PATH_INFO: `/v3/users/acf02b4c282b4f62853708452fa649e0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:41.404693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=acf02b4c282b4f62853708452fa649e0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:41.406118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:41.407704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 560/1121] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:41 2022] GET /identity/v3/users/acf02b4c282b4f62853708452fa649e0 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:41.413586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:41.477615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:41.511216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03db1ab-d49a-420b-85c8-29fbbc5c9365 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:41.552276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427dd2-aac7-441a-8e6e-02ddb2ce6378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:41.552911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427dd2-aac7-441a-8e6e-02ddb2ce6378 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:41.553281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427dd2-aac7-441a-8e6e-02ddb2ce6378 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:41.553455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427dd2-aac7-441a-8e6e-02ddb2ce6378 None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:41.554579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427dd2-aac7-441a-8e6e-02ddb2ce6378 None admin] RBAC: Authorizing `identity:get_project(project_id=43945b30ba42482cad079a3159431865)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:41.556064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427dd2-aac7-441a-8e6e-02ddb2ce6378 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:41.558228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 562/1122] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:41 2022] GET /identity/v3/projects/43945b30ba42482cad079a3159431865 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:41.564479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:41.618420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:41.666540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee26327-c6af-4dc3-ac4c-03b19f95f9e3 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:41.702246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812b8f7c-57d0-4034-98fc-8f9491d7fbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:41.702906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8f7c-57d0-4034-98fc-8f9491d7fbeb None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:41.703189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8f7c-57d0-4034-98fc-8f9491d7fbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:41.703465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812b8f7c-57d0-4034-98fc-8f9491d7fbeb None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865/users/acf02b4c282b4f62853708452fa649e0/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:41.705619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8f7c-57d0-4034-98fc-8f9491d7fbeb None admin] RBAC: Authorizing `identity:create_grant(project_id=43945b30ba42482cad079a3159431865, user_id=acf02b4c282b4f62853708452fa649e0, role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:41.707104 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812b8f7c-57d0-4034-98fc-8f9491d7fbeb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:41.720197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 561/1123] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:53:41 2022] PUT /identity/v3/projects/43945b30ba42482cad079a3159431865/users/acf02b4c282b4f62853708452fa649e0/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:53:43.588161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1eda0-29ab-421e-bd1c-1ec5fc7f009f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:43.588705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1eda0-29ab-421e-bd1c-1ec5fc7f009f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:43.588705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1eda0-29ab-421e-bd1c-1ec5fc7f009f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:43.590368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 563/1124] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:43.596051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:43.596521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:43.596614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:43.614998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:43.616213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:43.680692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 562/1125] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:43.953275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1eda0-29ab-421e-bd1c-1ec5fc7f009f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:44.053938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1eda0-29ab-421e-bd1c-1ec5fc7f009f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:44.087437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:44.088078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:44.088364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:44.088640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] PATH_INFO: `/v3/roles/30cc1d44cb1840079ee26860a103ecc6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:44.090933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] RBAC: Authorizing `identity:get_role(role_id=30cc1d44cb1840079ee26860a103ecc6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:44.092329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:44.093647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 564/1126] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:43 2022] GET /identity/v3/roles/30cc1d44cb1840079ee26860a103ecc6 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:44.099478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:44.141472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:44.197411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024c3dc-a0cd-467f-b4a0-0ad202805122 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:44.230735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:44.231289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:44.231567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:44.231838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] PATH_INFO: `/v3/users/acf02b4c282b4f62853708452fa649e0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:44.232979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] RBAC: Authorizing `identity:get_user(user_id=acf02b4c282b4f62853708452fa649e0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:44.234403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:44.235981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 563/1127] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:44 2022] GET /identity/v3/users/acf02b4c282b4f62853708452fa649e0 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:44.241823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:44.244724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:44.341803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df008bbb-e895-45c0-a6be-a92cc6dbecaf None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:44.376093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:44.376698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:44.376978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:44.377250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:44.442031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:44.443476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:44.445262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:44.540220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 565/1128] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:44.546865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:44.643997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be0e9e-5b99-433a-9f35-04d9e20f32e7 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:44.654381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:44.677731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09843e-cf24-404d-ae2a-8bdde4ca1bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:44.678349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09843e-cf24-404d-ae2a-8bdde4ca1bea None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:44.678625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09843e-cf24-404d-ae2a-8bdde4ca1bea None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:44.678893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09843e-cf24-404d-ae2a-8bdde4ca1bea None admin] PATH_INFO: `/v3/projects/43945b30ba42482cad079a3159431865` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:44.679986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09843e-cf24-404d-ae2a-8bdde4ca1bea None admin] RBAC: Authorizing `identity:get_project(project_id=43945b30ba42482cad079a3159431865)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:44.681431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09843e-cf24-404d-ae2a-8bdde4ca1bea None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:44.683734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 564/1129] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:44 2022] GET /identity/v3/projects/43945b30ba42482cad079a3159431865 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:44.689926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:44.786083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c4b74-c34d-48bb-9ec6-f30b116efc10 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:44.787623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:44.820203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:44.820726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:44.821003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:44.821273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:44.821991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] RBAC: Adding query filter params (role.id=30cc1d44cb1840079ee26860a103ecc6, scope.project.id=43945b30ba42482cad079a3159431865, user.id=acf02b4c282b4f62853708452fa649e0) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:44.822299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:44.824289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b88997f-0d0f-4ad9-81a7-4a807363727b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:44.829836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 566/1130] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:44 2022] GET /identity/v3/role_assignments?user.id=acf02b4c282b4f62853708452fa649e0&scope.project.id=43945b30ba42482cad079a3159431865&role.id=30cc1d44cb1840079ee26860a103ecc6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:46.781492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b43a9-b4e0-48c4-8e83-64acc191e4ea None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:46.782059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b43a9-b4e0-48c4-8e83-64acc191e4ea None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:46.782059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b43a9-b4e0-48c4-8e83-64acc191e4ea None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:46.783395 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 565/1131] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:46.788785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:46.789225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:46.789336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:46.807225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:46.808495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:46.873906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 567/1132] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 10:53:47.140437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b43a9-b4e0-48c4-8e83-64acc191e4ea None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:47.249423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b43a9-b4e0-48c4-8e83-64acc191e4ea None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:47.284378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd678a-b602-4886-90a1-8faf6251c8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:47.285007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd678a-b602-4886-90a1-8faf6251c8d8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:47.285286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd678a-b602-4886-90a1-8faf6251c8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:47.285556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd678a-b602-4886-90a1-8faf6251c8d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:47.286684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd678a-b602-4886-90a1-8faf6251c8d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:47.288146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd678a-b602-4886-90a1-8faf6251c8d8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:47.289947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 566/1133] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:47.296606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:47.392356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:47.399125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb61bd7-1768-43e9-baea-b2dbf8a0637e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:47.433839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e105bdd-da67-4457-969a-d69b8226960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:47.434443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105bdd-da67-4457-969a-d69b8226960a None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:47.434721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105bdd-da67-4457-969a-d69b8226960a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:47.434991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e105bdd-da67-4457-969a-d69b8226960a None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:47.435643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e105bdd-da67-4457-969a-d69b8226960a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:47.437105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e105bdd-da67-4457-969a-d69b8226960a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:47.465057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 568/1134] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:53:47 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:49.377691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc1e5e-bbd0-4697-bf24-d6d2f148d02e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:49.378260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc1e5e-bbd0-4697-bf24-d6d2f148d02e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:49.378260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc1e5e-bbd0-4697-bf24-d6d2f148d02e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:49.379617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 567/1135] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:49.387859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:49.388486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:49.388772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:49.407969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:49.409187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:49.476260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 569/1136] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:49.742829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc1e5e-bbd0-4697-bf24-d6d2f148d02e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:49.791234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:49.844661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc1e5e-bbd0-4697-bf24-d6d2f148d02e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:49.877133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad6272-a7b5-45ef-afb1-656cb2b2f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:49.877744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad6272-a7b5-45ef-afb1-656cb2b2f46e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:49.878020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad6272-a7b5-45ef-afb1-656cb2b2f46e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:49.878292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad6272-a7b5-45ef-afb1-656cb2b2f46e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:49.879387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad6272-a7b5-45ef-afb1-656cb2b2f46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:49.880831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad6272-a7b5-45ef-afb1-656cb2b2f46e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:49.882389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 568/1137] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:53:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:49.889376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:49.965626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:49.990034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74109a-12a0-475e-8f6d-82c01b66b3ac None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:50.023802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008ea38-7feb-4be4-9ccd-3ba7b3bf4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:50.024345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ea38-7feb-4be4-9ccd-3ba7b3bf4944 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:50.024623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ea38-7feb-4be4-9ccd-3ba7b3bf4944 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:50.024892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008ea38-7feb-4be4-9ccd-3ba7b3bf4944 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:50.025545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ea38-7feb-4be4-9ccd-3ba7b3bf4944 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:50.027553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008ea38-7feb-4be4-9ccd-3ba7b3bf4944 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:50.048911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 570/1138] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:53:49 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:53:52.031231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c190-8eba-49a8-a70b-56893b91e548 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:52.031815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c190-8eba-49a8-a70b-56893b91e548 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:52.031815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c190-8eba-49a8-a70b-56893b91e548 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:52.033201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 569/1139] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:52.038759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:52.039269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:52.039269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:52.057707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:52.059002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:52.125475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 571/1140] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:52.395037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85c190-8eba-49a8-a70b-56893b91e548 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:52.496052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85c190-8eba-49a8-a70b-56893b91e548 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:52.509604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:52.529983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:52.530604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:52.530889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:52.531160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:52.540964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:52.545814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:52.546993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:52.548130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 570/1141] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:53:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:52.553110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:52.613705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:52.651561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1df1ce-7fdd-49cd-8481-1577bbdb2d97 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:52.684902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:52.685501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:52.685780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:52.686051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:52.686715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:52.687020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:52.688412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:52.696304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 572/1142] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:52.702808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:52.801201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0004285b-e492-4b95-92f3-aa523d2ee080 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:52.833811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:52.834426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:52.834702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:52.834970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] PATH_INFO: `/v3/users/c127682408864e738dc9311fb9f4d9be` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:52.848033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] RBAC: Authorizing `identity:get_user(user_id=c127682408864e738dc9311fb9f4d9be)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:52.849465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:52.851067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 571/1143] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:52 2022] GET /identity/v3/users/c127682408864e738dc9311fb9f4d9be => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:52.857604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:52.958109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42fde9-781f-474b-a1d1-2542fa7c75d4 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:52.984754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:52.993761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:52.994372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:52.995220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:52.995487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:53.061668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:53.063126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:53.158659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 573/1144] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:53.164850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:53.196812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:53.271096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b8548-1fa8-40fb-bd7d-135e899c0c2a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:53.309331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d1df5-ccc5-4862-a846-97e831173fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:53.309953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d1df5-ccc5-4862-a846-97e831173fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:53.310232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d1df5-ccc5-4862-a846-97e831173fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:53.310507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d1df5-ccc5-4862-a846-97e831173fa2 None admin] PATH_INFO: `/v3/projects/e78e27a46871430695fe3a95c9070129` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:53.311614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d1df5-ccc5-4862-a846-97e831173fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=e78e27a46871430695fe3a95c9070129)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:53.313069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d1df5-ccc5-4862-a846-97e831173fa2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:53.314724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 572/1145] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:53 2022] GET /identity/v3/projects/e78e27a46871430695fe3a95c9070129 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:53.320870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:53.347646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:53.418168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb4095-c3ba-477c-9709-80974cd94916 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:53.451696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:53.452310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:53.452596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:53.452868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:53.453578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=e78e27a46871430695fe3a95c9070129, user.id=c127682408864e738dc9311fb9f4d9be) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:53.453885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:53.455292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b8567-4bd3-4ce6-9d95-3a820c838e4c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:53.460472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 574/1146] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:53 2022] GET /identity/v3/role_assignments?user.id=c127682408864e738dc9311fb9f4d9be&scope.project.id=e78e27a46871430695fe3a95c9070129&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:55.358709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412289e-0693-4e4e-9109-56370929df3d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:55.359272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412289e-0693-4e4e-9109-56370929df3d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:55.359272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412289e-0693-4e4e-9109-56370929df3d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:55.360618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 573/1147] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:55.365781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:55.366204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:55.366323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:55.384739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:55.386918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:55.451509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 575/1148] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:55.718913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412289e-0693-4e4e-9109-56370929df3d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:55.815808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:55.819934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412289e-0693-4e4e-9109-56370929df3d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:55.854834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:55.855457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:55.855736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:55.856035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:55.866078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:55.870993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:55.872101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:53:55.873180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 574/1149] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:55.878347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:55.899405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:55.974904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a98f9c-0d8b-45d1-a032-d185800ee5df None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:56.006697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:56.007286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:56.007562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:56.007833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:56.008515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:56.008817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:56.010708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:56.018414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 576/1150] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:56.024334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:56.122006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae880a-1bdc-4813-9173-e0842e72434d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:56.155120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:56.155741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:56.156043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:56.156314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] PATH_INFO: `/v3/users/c127682408864e738dc9311fb9f4d9be` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:56.157423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] RBAC: Authorizing `identity:get_user(user_id=c127682408864e738dc9311fb9f4d9be)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:56.158838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:56.160448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 575/1151] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:56 2022] GET /identity/v3/users/c127682408864e738dc9311fb9f4d9be => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:56.166440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:56.245987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:56.264828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcabac65-2793-4099-be48-58046bf1b200 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:56.297295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd41cc-53ba-4272-b454-de6bc643d31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:56.297915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd41cc-53ba-4272-b454-de6bc643d31e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:56.298189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd41cc-53ba-4272-b454-de6bc643d31e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:56.298460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd41cc-53ba-4272-b454-de6bc643d31e None admin] PATH_INFO: `/v3/projects/e78e27a46871430695fe3a95c9070129` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:56.299585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd41cc-53ba-4272-b454-de6bc643d31e None admin] RBAC: Authorizing `identity:get_project(project_id=e78e27a46871430695fe3a95c9070129)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:56.301030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd41cc-53ba-4272-b454-de6bc643d31e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:56.302646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 577/1152] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:56 2022] GET /identity/v3/projects/e78e27a46871430695fe3a95c9070129 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:56.309112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:56.337674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:56.407396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79425fc-d0d8-453c-8076-7229d0903d24 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:56.440360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa20fdf-c927-4647-9351-06fabfb295b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:56.441005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa20fdf-c927-4647-9351-06fabfb295b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:56.441284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa20fdf-c927-4647-9351-06fabfb295b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:56.441557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa20fdf-c927-4647-9351-06fabfb295b1 None admin] PATH_INFO: `/v3/projects/e78e27a46871430695fe3a95c9070129/users/c127682408864e738dc9311fb9f4d9be/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:56.443421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa20fdf-c927-4647-9351-06fabfb295b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=e78e27a46871430695fe3a95c9070129, user_id=c127682408864e738dc9311fb9f4d9be, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:56.444888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa20fdf-c927-4647-9351-06fabfb295b1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:56.458308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 576/1153] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:53:56 2022] PUT /identity/v3/projects/e78e27a46871430695fe3a95c9070129/users/c127682408864e738dc9311fb9f4d9be/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:53:58.368151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54af5-8b3f-4da8-be0a-9e0fba46a205 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:58.368683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54af5-8b3f-4da8-be0a-9e0fba46a205 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:58.368683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b54af5-8b3f-4da8-be0a-9e0fba46a205 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:58.370651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 578/1154] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:53:58.375775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:58.376231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:58.376364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:58.395335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:53:58.396559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:53:58.461810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 577/1155] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:53:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:53:58.732134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54af5-8b3f-4da8-be0a-9e0fba46a205 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:58.832822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b54af5-8b3f-4da8-be0a-9e0fba46a205 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:58.868721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:58.869354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:58.869616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:58.870029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:58.879783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:53:58.884564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:53:58.886301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:53:58.887383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 579/1156] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:53:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:53:58.892448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:58.910887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:58.992447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50488fb-680d-4a4a-ab0d-ac2ef7d11d6e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:59.025036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:59.026325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:59.026603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:59.026873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:59.027529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:59.027828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:59.029202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:59.037035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 578/1157] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:53:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:59.042850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:59.055336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:59.140324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9d814-e4aa-4c12-81a3-380a70351845 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:59.172991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:59.173605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:59.173879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:59.174148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] PATH_INFO: `/v3/users/c127682408864e738dc9311fb9f4d9be` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:59.175242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] RBAC: Authorizing `identity:get_user(user_id=c127682408864e738dc9311fb9f4d9be)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:59.176695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:59.178253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 580/1158] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:53:59 2022] GET /identity/v3/users/c127682408864e738dc9311fb9f4d9be => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:59.185232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:59.277726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:59.297200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a55d48-a7d2-47fc-a5da-7009511e8f4f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:59.329164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:59.329760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:59.330038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:59.330308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:59.395314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:59.396768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:59.474052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:59.493300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 579/1159] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:53:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:53:59.499440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:59.575584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:53:59.598005 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4905b-0206-4042-903a-ab99c93ea444 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:59.630042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f180328-9ee8-4777-9925-406a7e00370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:59.630651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180328-9ee8-4777-9925-406a7e00370d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:59.630925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180328-9ee8-4777-9925-406a7e00370d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:59.631193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f180328-9ee8-4777-9925-406a7e00370d None admin] PATH_INFO: `/v3/projects/e78e27a46871430695fe3a95c9070129` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:59.632385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180328-9ee8-4777-9925-406a7e00370d None admin] RBAC: Authorizing `identity:get_project(project_id=e78e27a46871430695fe3a95c9070129)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:59.633724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f180328-9ee8-4777-9925-406a7e00370d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:59.635861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 581/1160] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:53:59 2022] GET /identity/v3/projects/e78e27a46871430695fe3a95c9070129 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:53:59.642056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:53:59.737336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9deb33b-d51a-4520-848c-8f010434f1d8 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:53:59.773814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:53:59.774424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:53:59.774706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:53:59.774984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:53:59.776488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=e78e27a46871430695fe3a95c9070129, user.id=c127682408864e738dc9311fb9f4d9be) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:53:59.776842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:53:59.778489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c67f99-ba93-4270-85e9-a9af33829cf4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:53:59.785080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 580/1161] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:53:59 2022] GET /identity/v3/role_assignments?user.id=c127682408864e738dc9311fb9f4d9be&scope.project.id=e78e27a46871430695fe3a95c9070129&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:01.737920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87be46-14fa-4244-b308-e768a1aaf47e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:01.738528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87be46-14fa-4244-b308-e768a1aaf47e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:01.738528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87be46-14fa-4244-b308-e768a1aaf47e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:01.739814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 582/1162] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:01.745474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:01.745903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:01.746020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:01.764439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:01.765622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:01.806976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:01.833674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 581/1163] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:02.102730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87be46-14fa-4244-b308-e768a1aaf47e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:02.202354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87be46-14fa-4244-b308-e768a1aaf47e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:02.235544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:02.236171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:02.236451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:02.236721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:02.252015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:02.302329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:02.303781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:02.400900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 583/1164] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:54:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:54:02.406779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:02.504196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5507-1eef-49dd-933a-57ab65d78b19 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:02.537476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:02.538095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:02.538370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:02.538639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:02.545465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:02.546891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9b1b13ca-6275-400e-b09e-f0c451f5a32b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:54:02.557950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:54:02.559139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:54:02.560027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 582/1165] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:54:02 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:54:02.565123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:02.611500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:02.664503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d071441-adbd-4ce4-a86b-fa68fbb27a1d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:02.698308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:02.698916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:02.699195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:02.699463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:02.700143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:54:02.700449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:02.701825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204b9e8-46e9-4e1a-8a40-ac75be6e9c36 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:02.709163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 584/1166] 10.222.0.20 () {56 vars in 1115 bytes} [Wed Aug 31 10:54:02 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:04.596767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9c15-5751-4663-b60b-71e099a87f19 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:04.597325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9c15-5751-4663-b60b-71e099a87f19 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:04.597325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb9c15-5751-4663-b60b-71e099a87f19 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:04.598639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 583/1167] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:04.603685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b483-199b-486c-a8c5-51a2fefb5e5b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:04.604140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b483-199b-486c-a8c5-51a2fefb5e5b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:04.604250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0b483-199b-486c-a8c5-51a2fefb5e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:04.622394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-49d0b483-199b-486c-a8c5-51a2fefb5e5b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:04.623554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-49d0b483-199b-486c-a8c5-51a2fefb5e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:04.688669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 585/1168] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:04.954144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9c15-5751-4663-b60b-71e099a87f19 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:04.991222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:05.060894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb9c15-5751-4663-b60b-71e099a87f19 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:05.093854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbe048-acf6-48dc-98a5-bf77b70b4bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:05.094452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbe048-acf6-48dc-98a5-bf77b70b4bf2 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:05.094730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbe048-acf6-48dc-98a5-bf77b70b4bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:05.095002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbe048-acf6-48dc-98a5-bf77b70b4bf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:05.095674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbe048-acf6-48dc-98a5-bf77b70b4bf2 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:05.097085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edbe048-acf6-48dc-98a5-bf77b70b4bf2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:05.113568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 584/1169] 10.222.0.20 () {60 vars in 1138 bytes} [Wed Aug 31 10:54:04 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:54:07.082851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132ba4c-7943-437f-8ab3-2037d088006a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:07.083402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132ba4c-7943-437f-8ab3-2037d088006a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:07.083402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132ba4c-7943-437f-8ab3-2037d088006a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:07.084799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 586/1170] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:07.091655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:07.092152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:07.092449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:07.111360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:07.112591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:07.181544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 585/1171] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:07.452811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132ba4c-7943-437f-8ab3-2037d088006a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:07.496714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:07.557258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132ba4c-7943-437f-8ab3-2037d088006a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:07.591159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6864d-0230-4f14-9105-d27687911b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:07.591786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6864d-0230-4f14-9105-d27687911b85 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:07.592093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6864d-0230-4f14-9105-d27687911b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:07.592368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6864d-0230-4f14-9105-d27687911b85 None admin] PATH_INFO: `/v3/domains/19549237a0ad461b823ccc00ee9adbf5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:07.602607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6864d-0230-4f14-9105-d27687911b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=19549237a0ad461b823ccc00ee9adbf5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:07.604215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c6864d-0230-4f14-9105-d27687911b85 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:07.605840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 587/1172] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Aug 31 10:54:07 2022] GET /identity/v3/domains/19549237a0ad461b823ccc00ee9adbf5 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:07.611648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:07.636830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:07.712331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387d218-6d8b-4a90-add6-cab0c91acd34 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:07.746875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a90cb47-f2ee-4c7b-b8bd-57dae0a03172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:07.747472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90cb47-f2ee-4c7b-b8bd-57dae0a03172 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:07.747748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90cb47-f2ee-4c7b-b8bd-57dae0a03172 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:07.748039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a90cb47-f2ee-4c7b-b8bd-57dae0a03172 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:07.748698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90cb47-f2ee-4c7b-b8bd-57dae0a03172 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:07.750204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a90cb47-f2ee-4c7b-b8bd-57dae0a03172 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:07.777224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 586/1173] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:54:07 2022] POST /identity/v3/projects => generated 361 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:54:09.715032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6d37-d887-4e93-98e4-2de2eb5ca888 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:09.715596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6d37-d887-4e93-98e4-2de2eb5ca888 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:09.715596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6d37-d887-4e93-98e4-2de2eb5ca888 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:09.716957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 588/1174] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:09.722193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:09.722631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:09.722734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:09.742450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:09.744179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:09.812339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 587/1175] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:10.086825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6d37-d887-4e93-98e4-2de2eb5ca888 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:10.185490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:10.189769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6d37-d887-4e93-98e4-2de2eb5ca888 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:10.223475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1012cd7-0d4b-42e5-a4e4-e24408a7933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:10.224139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012cd7-0d4b-42e5-a4e4-e24408a7933d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:10.224421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012cd7-0d4b-42e5-a4e4-e24408a7933d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:10.224695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1012cd7-0d4b-42e5-a4e4-e24408a7933d None admin] PATH_INFO: `/v3/domains/19549237a0ad461b823ccc00ee9adbf5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:10.225804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1012cd7-0d4b-42e5-a4e4-e24408a7933d None admin] RBAC: Authorizing `identity:get_domain(domain_id=19549237a0ad461b823ccc00ee9adbf5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:10.227235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1012cd7-0d4b-42e5-a4e4-e24408a7933d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:10.229386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 589/1176] 10.222.0.20 () {56 vars in 1183 bytes} [Wed Aug 31 10:54:10 2022] GET /identity/v3/domains/19549237a0ad461b823ccc00ee9adbf5 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:10.235143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:10.243189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:10.336609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c184a22-269e-4550-93e2-a0dbe0c5f716 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:10.370918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7f760-5ef5-4d4a-9a8f-c26ab2541525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:10.371438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f760-5ef5-4d4a-9a8f-c26ab2541525 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:10.371711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f760-5ef5-4d4a-9a8f-c26ab2541525 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:10.371976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7f760-5ef5-4d4a-9a8f-c26ab2541525 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:10.372651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f760-5ef5-4d4a-9a8f-c26ab2541525 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:10.374062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7f760-5ef5-4d4a-9a8f-c26ab2541525 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:10.394214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 588/1177] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:54:10 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:54:12.443826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc25ab-0040-4603-8b2c-18ecc56a14a6 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:12.444417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc25ab-0040-4603-8b2c-18ecc56a14a6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:12.444417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc25ab-0040-4603-8b2c-18ecc56a14a6 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:12.445757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 590/1178] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:12.451000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:12.451437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:12.451546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:12.469714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:12.470899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:12.526878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:12.538026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 589/1179] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:12.808491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc25ab-0040-4603-8b2c-18ecc56a14a6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:12.906871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc25ab-0040-4603-8b2c-18ecc56a14a6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:12.940357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:12.940980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:12.941257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:12.941529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:12.952183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:54:12.956945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:54:12.958096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:54:12.959022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 591/1180] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:54:12.965523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:13.080264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d3306-5ab5-4623-a07a-d8974264d7ba None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:13.113736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:13.114336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:13.114613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:13.114883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:13.115537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:54:13.115841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:13.117222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:13.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 590/1181] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:54:13.131277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:13.160378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:13.230782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877af44-8bde-438f-a8fd-a827136c2c68 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:13.263734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:13.264384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:13.264665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:13.264939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] PATH_INFO: `/v3/users/2ac2539667594be3aa9838b0d3233748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:13.278578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] RBAC: Authorizing `identity:get_user(user_id=2ac2539667594be3aa9838b0d3233748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:13.280040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:13.281678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 592/1182] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:54:13 2022] GET /identity/v3/users/2ac2539667594be3aa9838b0d3233748 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:13.287904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:13.344287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:13.388987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c02fb-0120-4b7f-82a9-bc4884f58cb6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:13.421170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:13.421765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:13.422045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:13.422315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:13.487449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:13.488934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:13.541239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:13.585839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 591/1183] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:54:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:54:13.592283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:13.674181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:13.689355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78501a38-b788-4e76-8944-b5c197ab0bcd None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:13.723541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83824c-e223-4bae-9563-8916f25c37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:13.724196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83824c-e223-4bae-9563-8916f25c37f4 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:13.724475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83824c-e223-4bae-9563-8916f25c37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:13.724746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83824c-e223-4bae-9563-8916f25c37f4 None admin] PATH_INFO: `/v3/projects/14c129e034e44780bf7a47cf4dceb61c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:13.725852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83824c-e223-4bae-9563-8916f25c37f4 None admin] RBAC: Authorizing `identity:get_project(project_id=14c129e034e44780bf7a47cf4dceb61c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:13.727277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83824c-e223-4bae-9563-8916f25c37f4 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:13.729472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 593/1184] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:54:13 2022] GET /identity/v3/projects/14c129e034e44780bf7a47cf4dceb61c => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:13.735789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:13.833068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d956f1-97cb-4336-857d-bbb8cc8e96c4 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:13.866896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:13.867507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:13.867783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:13.868081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:13.869488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=14c129e034e44780bf7a47cf4dceb61c, user.id=2ac2539667594be3aa9838b0d3233748) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:54:13.869795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:13.871271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af2b154-964d-4cc7-8244-a12bae6dbc1f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:13.877301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 592/1185] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:54:13 2022] GET /identity/v3/role_assignments?user.id=2ac2539667594be3aa9838b0d3233748&scope.project.id=14c129e034e44780bf7a47cf4dceb61c&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:15.769940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0a3f-4e0e-4e02-81a9-210d3839a099 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:15.770496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0a3f-4e0e-4e02-81a9-210d3839a099 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:15.770496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd0a3f-4e0e-4e02-81a9-210d3839a099 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:15.771852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 594/1186] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:15.777920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:15.778472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:15.778472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:15.797129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:15.798300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:15.811486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:15.864406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 593/1187] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:16.135092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0a3f-4e0e-4e02-81a9-210d3839a099 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:16.236652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd0a3f-4e0e-4e02-81a9-210d3839a099 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:16.270422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:16.271041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:16.271322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:16.271597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:16.281637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:54:16.286427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:54:16.287521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:54:16.288801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 595/1188] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:54:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:54:16.294134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:16.392193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cd893-f5bf-4699-a567-724150bfbe51 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:16.425703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:16.426297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:16.426577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:16.426849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:16.427503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:54:16.427803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:16.429197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:16.437044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 594/1189] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:54:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:16.442894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:16.458868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:16.542551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08a418-851e-4239-b846-2af4b2a17634 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:16.574578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:16.575187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:16.575468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:16.575741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] PATH_INFO: `/v3/users/2ac2539667594be3aa9838b0d3233748` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:16.576869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] RBAC: Authorizing `identity:get_user(user_id=2ac2539667594be3aa9838b0d3233748)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:16.578296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:16.579867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 596/1190] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:54:16 2022] GET /identity/v3/users/2ac2539667594be3aa9838b0d3233748 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:16.586174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:16.599266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:16.684842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e60b8-1360-4016-bcf7-8d3e45ea3084 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:16.717787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4cfbf-564e-4af8-8f3a-d67e401ac574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:16.718402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cfbf-564e-4af8-8f3a-d67e401ac574 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:16.718678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cfbf-564e-4af8-8f3a-d67e401ac574 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:16.718946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4cfbf-564e-4af8-8f3a-d67e401ac574 None admin] PATH_INFO: `/v3/projects/14c129e034e44780bf7a47cf4dceb61c` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:16.720089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4cfbf-564e-4af8-8f3a-d67e401ac574 None admin] RBAC: Authorizing `identity:get_project(project_id=14c129e034e44780bf7a47cf4dceb61c)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:16.721507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4cfbf-564e-4af8-8f3a-d67e401ac574 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:16.723126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 595/1191] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:54:16 2022] GET /identity/v3/projects/14c129e034e44780bf7a47cf4dceb61c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:54:16.729194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:16.809426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:16.827730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6c2920-9ddf-49f5-9f82-8f82077eca2a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:16.861210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b38cfa-97d0-4492-87f0-3ef91adffc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:16.861857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38cfa-97d0-4492-87f0-3ef91adffc23 None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:16.862135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38cfa-97d0-4492-87f0-3ef91adffc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:16.862505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b38cfa-97d0-4492-87f0-3ef91adffc23 None admin] PATH_INFO: `/v3/projects/14c129e034e44780bf7a47cf4dceb61c/users/2ac2539667594be3aa9838b0d3233748/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:16.864419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b38cfa-97d0-4492-87f0-3ef91adffc23 None admin] RBAC: Authorizing `identity:create_grant(project_id=14c129e034e44780bf7a47cf4dceb61c, user_id=2ac2539667594be3aa9838b0d3233748, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:16.865878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b38cfa-97d0-4492-87f0-3ef91adffc23 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:16.879258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 597/1192] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:54:16 2022] PUT /identity/v3/projects/14c129e034e44780bf7a47cf4dceb61c/users/2ac2539667594be3aa9838b0d3233748/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:54:18.778015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912710f5-402c-4835-bfa6-e3a403e15657 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:18.778544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912710f5-402c-4835-bfa6-e3a403e15657 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:18.778544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912710f5-402c-4835-bfa6-e3a403e15657 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:18.779895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 596/1193] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:54:18.785437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:18.785806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:18.785982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:18.805231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:54:18.806943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:54:18.873363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 598/1194] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:54:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:54:19.139270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912710f5-402c-4835-bfa6-e3a403e15657 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:19.216826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:19.239161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912710f5-402c-4835-bfa6-e3a403e15657 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:19.271748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:19.272386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:19.272663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:19.272934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:19.283383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:54:19.288126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:54:19.289243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:54:19.290634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 597/1195] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:54:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:54:19.295967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:19.368361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:19.397376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab12581-a363-4d21-aa72-d6dc4e1e3d9d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:19.432114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:19.432727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:19.433009 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:19.433284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:19.433996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:54:19.434305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:19.435694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:19.444351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 599/1196] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:54:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:19.450286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:19.548032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de791bf-561e-40f5-87d0-350d86cd8552 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:19.580095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:19.580709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:19.580985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:19.581253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] PATH_INFO: `/v3/users/2ac2539667594be3aa9838b0d3233748` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:19.582351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac2539667594be3aa9838b0d3233748)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:19.583760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:19.585628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 598/1197] 10.222.0.20 () {56 vars in 1177 bytes} [Wed Aug 31 10:54:19 2022] GET /identity/v3/users/2ac2539667594be3aa9838b0d3233748 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:19.592112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:19.691715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83c0949-ad52-4572-908a-217548066c2f None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:19.727147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:19.727780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:19.728246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:19.728522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:19.731146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:19.796846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:19.798295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:19.898025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 600/1198] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:54:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:54:19.904943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:19.915468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:20.010197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e0fded-921a-49db-b4d9-797326c80a20 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:20.046210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287d053-a528-42c5-9562-a11ababbb65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:20.046833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287d053-a528-42c5-9562-a11ababbb65a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:20.047110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287d053-a528-42c5-9562-a11ababbb65a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:20.047382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287d053-a528-42c5-9562-a11ababbb65a None admin] PATH_INFO: `/v3/projects/14c129e034e44780bf7a47cf4dceb61c` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:20.048539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287d053-a528-42c5-9562-a11ababbb65a None admin] RBAC: Authorizing `identity:get_project(project_id=14c129e034e44780bf7a47cf4dceb61c)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:20.049967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287d053-a528-42c5-9562-a11ababbb65a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:20.051623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 599/1199] 10.222.0.20 () {56 vars in 1186 bytes} [Wed Aug 31 10:54:19 2022] GET /identity/v3/projects/14c129e034e44780bf7a47cf4dceb61c => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:20.058002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:54:20.094161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:20.160941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef83b0-b346-4c48-ad83-6551d32eb5de None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:54:20.195100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:54:20.195698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:54:20.196052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:54:20.196884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:54:20.197622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=14c129e034e44780bf7a47cf4dceb61c, user.id=2ac2539667594be3aa9838b0d3233748) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:54:20.197930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:54:20.199358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937a2ac-331e-4572-a217-8673cb11d862 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:54:20.205203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 601/1200] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:54:20 2022] GET /identity/v3/role_assignments?user.id=2ac2539667594be3aa9838b0d3233748&scope.project.id=14c129e034e44780bf7a47cf4dceb61c&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:54:30.042405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=63 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:54:30.202541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=51 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:16.213231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f13da-9d95-477f-90ba-c270e292ecae None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:16.213871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f13da-9d95-477f-90ba-c270e292ecae None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:16.213871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f13da-9d95-477f-90ba-c270e292ecae None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:16.215417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 600/1201] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:16.220728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:16.221236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:16.221327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:16.248915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:16.250215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:16.328934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 602/1202] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:16.672488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f13da-9d95-477f-90ba-c270e292ecae None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:16.805376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f13da-9d95-477f-90ba-c270e292ecae None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:16.851999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:16.852682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:16.852968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:16.853244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:16.869405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:16.871037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:16.891745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:16.898984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:16.899963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:16.899963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:16.899963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:16.899963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 601/1203] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:16.900254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:17.038731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b9441-4c5d-45da-9c7b-dbd40142f901 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:17.090275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:17.098270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:17.099228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:17.099579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:17.100088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:17.100769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:17.101149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:17.102785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca961b-8b78-42c3-ae1d-9bcd595bba68 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:17.118849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 603/1204] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:17.125787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:17.200275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:17.262874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50bc7c-354a-4ba3-93cf-0ab80de8c8f6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:17.313335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18b9d2-dad7-4cdc-83b6-d3b8cb0b7508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:17.313970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b9d2-dad7-4cdc-83b6-d3b8cb0b7508 None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:17.314239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b9d2-dad7-4cdc-83b6-d3b8cb0b7508 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:17.314562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18b9d2-dad7-4cdc-83b6-d3b8cb0b7508 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:17.315187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b9d2-dad7-4cdc-83b6-d3b8cb0b7508 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:17.316798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe18b9d2-dad7-4cdc-83b6-d3b8cb0b7508 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:17.345882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 602/1205] 10.222.0.20 () {60 vars in 1132 bytes} [Wed Aug 31 10:55:17 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:55:19.572172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a979283-22f1-415a-9f82-7e09487bc96c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:19.572724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a979283-22f1-415a-9f82-7e09487bc96c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:19.572724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a979283-22f1-415a-9f82-7e09487bc96c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:19.574215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 604/1206] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:19.579360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:19.579770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:19.580051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:19.601882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:19.603587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:19.680047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 603/1207] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:19.956035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a979283-22f1-415a-9f82-7e09487bc96c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:20.067240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a979283-22f1-415a-9f82-7e09487bc96c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:20.104749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:20.105441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.105699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.105968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.117790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:55:20.124300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:20.125264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:20.131492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 605/1208] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:55:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:20.132266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:20.176680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:20.242956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c582fe5-f335-432c-a940-f0b3b943b4f0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:20.278823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:20.279427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.279705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.279973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.280678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:20.281014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:20.282325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:20.291218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 604/1209] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:55:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:55:20.296933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:20.351772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:20.423207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd5a46-ef64-4115-8550-4e05bee2b10a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:20.463895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:20.464684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.464963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.465232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.478143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:20.480038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:20.498750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:20.499775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:20.502199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:20.502199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:20.502199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:20.505823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 606/1210] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:20.507793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:20.617920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:20.621415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409ef95-a8dd-48e6-8c87-c93ed414fe22 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:20.665252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:20.665924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.666291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.666851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.667500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:20.667802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:20.669302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:20.683557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 605/1211] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:55:20.689969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:20.808545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00447e-0ff1-435f-978f-9179667fd6cb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:20.810186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:20.869292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:20.869994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:20.870381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:20.870700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:20.882822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:20.882822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:55:20.896102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:20.897310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:20.898288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 607/1212] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:20.903275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:21.029194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e63c526-02e7-4d13-a95a-6ec98b00fd53 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:21.071129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:21.071615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:21.071988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:21.072300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:21.072934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:21.073239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:21.075208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:21.094103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 606/1213] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 10:55:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:21.100353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:21.226998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e781-e5fe-42be-98b4-baaa79b97bca None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:21.267479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:21.268166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:21.268536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:21.268715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:21.305404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:21.351516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:21.353207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:21.488039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 608/1214] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:55:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:55:21.494902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:21.519609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:21.615469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94960b-6a30-4ac2-b515-58547c9a8c34 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:21.655734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:21.658101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:21.658101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:21.658101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:21.669430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:21.670972 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:21.686524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:21.687569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:21.693783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:21.693783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:21.693783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:21.693783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 607/1215] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:21.694030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:21.761713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:21.802057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7b297-85f4-4a78-9c42-03640d10172b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:21.845703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:21.846392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:21.846695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:21.846968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:21.847587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:21.847918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:21.851045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:21.868035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 609/1216] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:21.869232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:21.955788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:21.995213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa885d-de91-420a-88c3-d45ef6a0596e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:22.032374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:22.033003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:22.033320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:22.033593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:22.040995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:22.042491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:55:22.055115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:22.061614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:22.062567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 608/1217] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:22.063084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:22.179937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c36d58a-a111-4dd0-ac1b-2e6bed73ed0a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:22.204900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:22.218709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:22.219452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:22.219839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:22.220210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:22.221000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:22.221400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:22.222911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a29ec1-ea6e-40b0-8171-270b178da6c2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:22.236113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 610/1218] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:55:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:22.244544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:22.374636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffee049-a705-46fa-8e9e-622d748c6c5d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:22.407642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:22.414330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:22.415003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:22.415284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:22.415560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:22.416351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=f5103a11f8f446778043577e13cea822) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:22.416665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:22.418178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8713bc-15f5-4a4d-8656-f55e8e183ef9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:22.424538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 609/1219] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:55:22 2022] GET /identity/v3/role_assignments?user.id=f5103a11f8f446778043577e13cea822&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:55:24.385032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6c861-cfaa-4cc9-b1e9-9af48fca481c None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:24.385573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6c861-cfaa-4cc9-b1e9-9af48fca481c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:24.385573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6c861-cfaa-4cc9-b1e9-9af48fca481c None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:24.386959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 611/1220] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:24.392362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:24.392788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:24.392906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:24.413688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:24.414890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:24.491486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 610/1221] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:24.762127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6c861-cfaa-4cc9-b1e9-9af48fca481c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:24.877476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6c861-cfaa-4cc9-b1e9-9af48fca481c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:24.914008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:24.914642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:24.914917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:24.915187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:24.925380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:55:24.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:24.936120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:24.937083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 612/1222] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:55:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:24.938445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:25.050719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc0ad3-407a-4e05-818b-4040ecbd7725 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:25.076277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:25.089472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:25.090122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:25.090407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:25.090715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:25.091469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] RBAC: Adding query filter params (name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:25.091821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:25.093391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:25.102065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 611/1223] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:55:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:25.107909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:25.158164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:25.222629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e254b6f-632d-4bf6-ba71-10ccc92b07ef None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:25.259119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:25.259748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:25.260112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:25.260377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:25.271954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:25.273451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:25.288865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:25.290035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:25.291197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:25.291197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:25.291197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:25.291197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 613/1224] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:25.296495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:25.406409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291ec740-7321-4266-985d-a7678b683fdf None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:25.446399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:25.447004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:25.447282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:25.447555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:25.448244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:25.448540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:25.450058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:25.462968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 612/1225] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:25.468696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:25.570823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:25.578451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3a65-77db-4db7-8b2a-93b9d43c8b4d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:25.614861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:25.615708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:25.616188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:25.617449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:25.627964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:25.629435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:55:25.642634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:25.648242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:25.649620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 614/1226] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:25.650387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:25.704271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:25.763498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59864993-445e-4d3b-943d-406573e46e29 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:25.803955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:25.804589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:25.804867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:25.805135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:25.806020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:25.806318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:25.808129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:25.827920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 613/1227] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 10:55:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:25.834828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:25.953593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0fae8-ab3b-42fe-8433-8b929070cecb None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:25.993728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:25.994288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:25.994605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:25.994878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.007495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:26.009468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:26.029704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:26.030961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:26.032079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:26.032079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:26.032079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:26.036501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 615/1228] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:26.037869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:26.148511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792918bd-89be-4ffa-9fdb-af61c6dd3495 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:26.151375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:26.188669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:26.189501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:26.189787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:26.190095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.190940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:26.191428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:26.193687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:26.208843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 614/1229] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:55:26.215104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:26.225053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:26.328973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6214618e-6943-45c4-87b3-fcac6a30d1f5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:26.369059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:26.369732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:26.370068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:26.370375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.378221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:26.379708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:55:26.391907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:26.393841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:26.396547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 616/1230] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:26.402118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:26.522468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79939da-4c34-4c9c-bcd4-708906232fca None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:26.557965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:26.558602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:26.558879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:26.559146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.559871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:26.560200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:26.561698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc6eacc-42fb-45e2-a94b-0342de837541 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:26.575188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 615/1231] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:55:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:26.581332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:26.652294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:26.691368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b27ec5-7b93-440d-93d3-884a5768d37c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:26.729497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12f8a9-9589-4ac6-9cee-edfe3ad59b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:26.730162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f8a9-9589-4ac6-9cee-edfe3ad59b4d None admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:26.730445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f8a9-9589-4ac6-9cee-edfe3ad59b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:26.730728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12f8a9-9589-4ac6-9cee-edfe3ad59b4d None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/f5103a11f8f446778043577e13cea822/roles/cef6888092d14f27858a8cb3c8659eb0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:26.749675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12f8a9-9589-4ac6-9cee-edfe3ad59b4d None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=f5103a11f8f446778043577e13cea822, role_id=cef6888092d14f27858a8cb3c8659eb0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:26.751147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12f8a9-9589-4ac6-9cee-edfe3ad59b4d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:26.765371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 617/1232] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:55:26 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/f5103a11f8f446778043577e13cea822/roles/cef6888092d14f27858a8cb3c8659eb0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:55:28.762978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5012e-7515-45e3-aec0-136f85014895 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:28.763541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5012e-7515-45e3-aec0-136f85014895 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:28.763541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee5012e-7515-45e3-aec0-136f85014895 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:28.765088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 616/1233] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:28.771998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:28.772493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:28.772601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:28.794278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:28.795494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:28.872603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 618/1234] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:29.150223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5012e-7515-45e3-aec0-136f85014895 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:29.177357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:29.264320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee5012e-7515-45e3-aec0-136f85014895 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:29.300337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:29.300990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:29.302087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:29.302203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:29.313533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:55:29.319165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:29.324113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:29.325232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 617/1235] 10.222.0.20 () {56 vars in 1102 bytes} [Wed Aug 31 10:55:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:29.326589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:29.378264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:29.436990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5fc97-c952-49cc-b739-8abdb13c2414 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:29.473838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:29.475038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:29.475332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:29.475700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:29.476385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] RBAC: Adding query filter params (name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:29.476705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:29.478238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:29.487224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 619/1236] 10.222.0.20 () {56 vars in 1103 bytes} [Wed Aug 31 10:55:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:29.494441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:29.586987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:29.605513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8d00c-394d-4a05-b4af-18922566e83d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:29.641123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:29.641756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:29.642030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:29.642314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:29.654285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:29.655782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:29.671233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:29.677938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:29.678915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:29.678915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:29.678915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:29.678915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 618/1237] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:29.679146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:29.796904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5255191-fad4-4b9f-be72-941d34b43564 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:29.807014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:29.835543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:29.836223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:29.836529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:29.836799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:29.837442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:29.837739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:29.839104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:29.851910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 620/1238] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:29.857995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:29.972289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95237909-eb65-421a-9312-978ffce01fb0 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:30.018204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:30.018828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:30.019134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:30.019442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:30.023684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:30.030591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:30.032534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:55:30.048612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:30.055670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:30.056659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 619/1239] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:30.057390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:30.166045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56beb387-77fd-41bd-9e35-24cb3a69b77a None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:30.209763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:30.210583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:30.210938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:30.211220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:30.212060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:30.212373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:30.214425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:30.242168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 621/1240] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 10:55:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:30.243510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:30.357707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfeb27-8f35-4249-b7a6-34611de653bd None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:30.393375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:30.394041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:30.394355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:30.394693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:30.460305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:30.468474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:30.470165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:30.580831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 620/1241] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:55:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:55:30.587808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:30.618306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:30.697443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430fd870-f0de-4834-af67-8bc645a7a368 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:30.735746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:30.736490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:30.736807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:30.737064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:30.749114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:30.750684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:30.768735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:30.770360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:30.771405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:30.771405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:30.771405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:30.771405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 622/1242] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:30.776472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:30.877158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:30.885383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efeb32f-5665-4557-b012-e02ddfb74464 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:30.925269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:30.925990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:30.926811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:30.927171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:30.927903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:30.928233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:30.929728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:30.943066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 621/1243] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:30.948669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:31.043831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:31.069476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f50030-38be-4cec-8228-a9bd90eecda5 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:31.105621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:31.106265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:31.106544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:31.106812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:31.115275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:31.116914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:55:31.130215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:31.131338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:31.132723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 623/1244] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:55:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:31.137435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:31.245324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf12d903-d711-4196-8a85-dd0cef2f09a3 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:31.282196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:31.282921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:31.283278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:31.283660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:31.284475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:31.284886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:31.286748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635df1b7-a38c-42d5-a019-b4ee64c69344 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:31.299522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 622/1245] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:31.305962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:31.423271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9411b2-f7fd-4bc7-a60f-bb8803d12e62 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:31.461770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:31.462388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:31.462654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:31.463063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:31.463793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] RBAC: Adding query filter params (role.id=cef6888092d14f27858a8cb3c8659eb0, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=f5103a11f8f446778043577e13cea822) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:31.464164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:31.465661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887904e-2c78-441f-818d-2e8e8e648755 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:31.471896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 624/1246] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:55:31 2022] GET /identity/v3/role_assignments?user.id=f5103a11f8f446778043577e13cea822&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=cef6888092d14f27858a8cb3c8659eb0 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:33.506449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4863cb8-59cc-481a-bffb-664afa91df28 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:33.507118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4863cb8-59cc-481a-bffb-664afa91df28 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:33.507118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4863cb8-59cc-481a-bffb-664afa91df28 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:33.508607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 623/1247] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:33.514086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:33.514562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:33.514668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:33.522970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:33.537410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:33.538613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:33.616898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 625/1248] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:33.896350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4863cb8-59cc-481a-bffb-664afa91df28 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:33.917171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:34.017179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4863cb8-59cc-481a-bffb-664afa91df28 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:34.061320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:34.061886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:34.062167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:34.062443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:34.075182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:55:34.080873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:34.082423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:34.083318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 624/1249] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:55:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:34.090213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:34.203269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3963a7-85db-4b31-afea-95be9f7a37a8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:34.237953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:34.244316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:34.244932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:34.245217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:34.245495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:34.246415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:34.246767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:34.248305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:34.257771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 626/1250] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:55:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:34.263570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:34.358843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:34.385343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab67b1e-a488-4c72-8718-27f4dd982238 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:34.425420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:34.426533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:34.426966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:34.427301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:34.438929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:34.440485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:34.459312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:34.460574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:34.461811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:34.461811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:34.461811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:34.461811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 625/1251] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:34.466751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:34.584225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706c369-1b09-4bfc-a3b6-a3b63e88f54d None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:34.625912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:34.626492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:34.626776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:34.627055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:34.627714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:34.628058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:34.629562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:34.644527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 627/1252] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:34.651569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:34.771315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744eac61-ef3b-495c-bd6f-51118a28cbf9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:34.801825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:34.811116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:34.811770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:34.812062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:34.812342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:34.821605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:34.823209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:55:34.835664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:34.836974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:34.838342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 626/1253] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:34.843115 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:34.904843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:34.955656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b01c84-e55a-4a20-9823-e178c436fff1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:35.001113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:35.001853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.002134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.002417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.003243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:35.003641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:35.005765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:35.030569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 628/1254] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 10:55:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:35.037287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:35.146222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1b5e2-fc2c-4e81-b49f-975abd3a5412 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:35.183629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:35.184187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.184472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.184745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.252921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:35.266934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:35.268424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:35.376503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 627/1255] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:55:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 10:55:35.383232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:35.497794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d9186-1d2b-4ddb-b103-5f16230a0368 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:35.512797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:35.536210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:35.536857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.537139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.537416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.547768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:35.549253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:35.564164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:35.565211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:35.567386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:35.567386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:35.567386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:35.567386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 629/1256] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:35.571725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:35.671062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:35.685285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e54ce-da13-4c2b-9bb8-6bb4c2f53595 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:35.721457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:35.722073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.722344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.722616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.723770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:35.724102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:35.725568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:35.739576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 628/1257] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:35.745269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:35.857610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20223b3f-9a56-4eee-8612-2034c48c67d1 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:35.893097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:35.893761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:35.894107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:35.894374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:35.903131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:35.904820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:55:35.917494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:35.918666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:35.919669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 630/1258] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:55:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:35.925729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:36.043118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f4805-e3db-4f55-9b5d-c9e7caf5577f None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:36.078991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:36.079594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:36.079866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:36.080164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:36.080854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:36.081423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:36.083376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314ebac-ac63-4328-991f-e01adae06e03 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:36.096813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 629/1259] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:36.103717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:36.130303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:36.217018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ea3f9-dc71-4971-a1a2-9461127eb7af None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:36.255659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:36.256378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:36.256739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:36.257045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:36.257775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=f5103a11f8f446778043577e13cea822) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:36.258084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:36.259631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a850238-4501-4adb-9fe6-2ab10cbfcfc9 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:36.265612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 631/1260] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:55:36 2022] GET /identity/v3/role_assignments?user.id=f5103a11f8f446778043577e13cea822&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:38.218322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a0f4c-4018-4117-a530-ffe3b2a3677e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:38.218949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a0f4c-4018-4117-a530-ffe3b2a3677e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:38.218949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a0f4c-4018-4117-a530-ffe3b2a3677e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:38.220250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 630/1261] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:38.225901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:38.226426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:38.226426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:38.245959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:38.247149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:38.316900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 632/1262] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:38.588886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a0f4c-4018-4117-a530-ffe3b2a3677e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:38.600220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:38.691437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a0f4c-4018-4117-a530-ffe3b2a3677e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:38.724492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:38.725097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:38.725371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:38.725638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:38.735257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:55:38.740626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:38.741835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:38.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 631/1263] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:55:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:38.747879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:38.766103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:38.850419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d0ff4a-5efc-451f-9a86-a6eb23dd5909 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:38.886015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:38.886645 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:38.886922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:38.887247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:38.888040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:38.888341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:38.889869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:38.899870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 633/1264] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:55:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:38.905651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:38.975653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:39.012171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739b062-e3ca-46ad-a4aa-ad9e09f1ac4d None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:39.049140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:39.049760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:39.050037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:39.050306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:39.060832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:39.062294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:39.076979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:39.082566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:39.083764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:39.083764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:39.083764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:39.083764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 632/1265] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:39.084972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:39.167394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:39.192839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480824-8f1d-45e4-9d11-f9c80462e719 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:39.226320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:39.226995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:39.227185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:39.227458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:39.228132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:39.228440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:39.229799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:39.241934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 634/1266] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:39.247681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:39.347269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c849f4d-324a-4b4b-b35b-951372db13fb None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:39.367767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:39.381803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:39.382419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:39.382700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:39.382971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:39.391059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:39.392560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:55:39.405478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:39.406712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:39.407992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 633/1267] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:39.413285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:39.514420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c514218a-b2a5-4edb-87ed-d5dcd0fbdd08 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:39.554670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:39.555274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:39.555555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:39.555828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:39.556583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:39.556887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:39.558315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:39.567805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:39.577209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 635/1268] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 10:55:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:39.583517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:39.690846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15befcae-6663-40b0-a6e4-8e268e5707e2 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:39.730445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:39.731080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:39.731354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:39.731618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:39.744303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:39.745745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:39.761409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:39.762601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:39.763661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:39.763661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:39.763661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:39.763661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 634/1269] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:39.768561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:39.893842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f3c7e4-2695-4cc1-b1f0-274fb8c49442 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:39.931066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:39.931740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:39.932050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:39.932322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:39.932964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:39.933267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:39.934644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:39.947245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 636/1270] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:39.953135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:39.991521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:40.064779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ca8e6-dd86-4d9f-9799-ea4fe3a28ac9 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:40.098260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:40.098894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:40.099162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:40.099426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:40.106460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:40.107952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:55:40.121540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:40.122670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:40.123983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 635/1271] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:55:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:40.128589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:40.188577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:40.235327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e4f5b3-3b74-424a-bed8-0faf4a913d69 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:40.268823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:40.269550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:40.270013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:40.270292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:40.270982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:40.271287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:40.272742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1df287b-e5fb-48b1-b3fa-c1895ca2daa2 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:40.284521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 637/1272] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:40.291163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:40.382491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:40.391725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88932c13-e146-4bd5-8a8b-3424b565291a None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:40.424884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34cc07-7e2d-4380-84a3-203faf547b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:40.425538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34cc07-7e2d-4380-84a3-203faf547b16 None admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:40.426349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34cc07-7e2d-4380-84a3-203faf547b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:40.426625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34cc07-7e2d-4380-84a3-203faf547b16 None admin] PATH_INFO: `/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/f5103a11f8f446778043577e13cea822/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:40.428528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34cc07-7e2d-4380-84a3-203faf547b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a0a65f6805f4a5996e5fccef0bcf57e, user_id=f5103a11f8f446778043577e13cea822, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:40.429950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34cc07-7e2d-4380-84a3-203faf547b16 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:40.443393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 636/1273] 10.222.0.20 () {58 vars in 1439 bytes} [Wed Aug 31 10:55:40 2022] PUT /identity/v3/projects/3a0a65f6805f4a5996e5fccef0bcf57e/users/f5103a11f8f446778043577e13cea822/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 10:55:42.383196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d93ed-9d5e-4e74-b279-f07e2c21639d None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:42.383769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d93ed-9d5e-4e74-b279-f07e2c21639d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:42.383769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d93ed-9d5e-4e74-b279-f07e2c21639d None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:42.385106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 638/1274] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:42.390383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:42.390838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:42.390942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:42.410975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:42.412244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:42.481668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 637/1275] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:42.758250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d93ed-9d5e-4e74-b279-f07e2c21639d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:42.858581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d93ed-9d5e-4e74-b279-f07e2c21639d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:42.876345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:42.903659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:42.904328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:42.904609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:42.904883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:42.914867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 10:55:42.919718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:42.920859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:42.921886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 639/1276] 10.222.0.20 () {56 vars in 1096 bytes} [Wed Aug 31 10:55:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:42.927401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:43.011524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:43.030665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad4e43-1bf4-4c3b-b027-bee03b6ec76b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:43.067211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:43.067806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:43.068112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:43.068587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:43.069329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:43.069631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:43.071288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:43.079718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 638/1277] 10.222.0.20 () {56 vars in 1099 bytes} [Wed Aug 31 10:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:43.087730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:43.192491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d642d-c8b3-4f18-b027-c8b26c0b4e74 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:43.226430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:43.227049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:43.227330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:43.227603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:43.237305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:43.239158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:43.253295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:43.255182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:43.256143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:43.256143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:43.256143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:43.256143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 640/1278] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 10:55:43.261313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:43.358826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fe556-89e8-4ac0-9b0a-07121b3dbec6 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:43.392425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:43.393063 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:43.393280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:43.393548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:43.394181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:43.394680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:43.396051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:43.407785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 639/1279] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:43.413748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:43.445475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:43.514202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462ed9d-263f-4d84-8ca0-395489ea81a0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:43.547222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:43.547843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:43.548157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:43.548436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:43.556407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:43.557864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 10:55:43.569440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:43.570535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:43.571677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 641/1280] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:43.576412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:43.580158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:43.674597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed961f-6117-4c01-a2da-1a4d7a485115 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:43.714723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:43.715330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:43.715611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:43.715883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:43.716646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:43.716950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:43.718369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:43.737470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 640/1281] 10.222.0.20 () {56 vars in 1143 bytes} [Wed Aug 31 10:55:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:43.746772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:43.844840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:43.854652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ccd61-3090-46b3-bd1d-7881a2d4a296 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:43.888054 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adda37-833d-452d-9366-796abed73e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:43.888656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adda37-833d-452d-9366-796abed73e9b None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:43.888935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adda37-833d-452d-9366-796abed73e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:43.889209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adda37-833d-452d-9366-796abed73e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:43.955426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adda37-833d-452d-9366-796abed73e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:43.957880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adda37-833d-452d-9366-796abed73e9b None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:44.051525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:44.062164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 642/1282] 10.222.0.20 () {58 vars in 1282 bytes} [Wed Aug 31 10:55:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 10:55:44.068862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:44.134834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:44.172612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62202825-f6c8-4e64-a1d3-5370374cee0e None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:44.206707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:44.207231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:44.207508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:44.207774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:44.217492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:44.218922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 10:55:44.233026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:44.234248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 10:55:44.235552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 10:55:44.235552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 10:55:44.235552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:44.235552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 641/1283] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 10:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:44.240617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adda37-833d-452d-9366-796abed73e9b None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:44.335070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adda37-833d-452d-9366-796abed73e9b None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:44.370008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:44.370604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:44.370878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:44.371147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:44.371777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:44.372104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:44.373460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:44.384806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 643/1284] 10.222.0.20 () {56 vars in 1109 bytes} [Wed Aug 31 10:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:44.390551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:44.496477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f901-f841-428e-aef3-b89147af2181 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:44.510111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:44.530311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:44.530931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:44.531210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:44.531483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:44.538584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:44.540024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 10:55:44.551687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:44.552772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: ERROR keystone.server.flask.application Aug 31 10:55:44.554031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 642/1285] 10.222.0.20 () {56 vars in 1111 bytes} [Wed Aug 31 10:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:44.558727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:44.607765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:44.656846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594d1efb-cb66-456c-ab27-65eb641cb059 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:44.691494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:44.692126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:44.692407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:44.692677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:44.693373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:44.693679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:44.695087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67d095-a673-49df-b9f3-2679536a8bc0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:44.707571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 644/1286] 10.222.0.20 () {56 vars in 1148 bytes} [Wed Aug 31 10:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:44.713791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:44.816706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499bc32-87a2-4341-8370-1bc06cd46077 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:44.849503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:44.850101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:44.850369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:44.850642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:44.851358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] RBAC: Adding query filter params (role.id=e6bfcb3b509648a0888a6485214b8ccb, scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e, user.id=f5103a11f8f446778043577e13cea822) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:44.851666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:44.853112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6320bcc-846c-489c-9574-56b0681fe674 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:44.858720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 643/1287] 10.222.0.20 () {56 vars in 1374 bytes} [Wed Aug 31 10:55:44 2022] GET /identity/v3/role_assignments?user.id=f5103a11f8f446778043577e13cea822&scope.project.id=3a0a65f6805f4a5996e5fccef0bcf57e&role.id=e6bfcb3b509648a0888a6485214b8ccb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:46.816320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2633ee-62e3-45c3-9133-ca2a633d882a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:46.816888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2633ee-62e3-45c3-9133-ca2a633d882a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:46.816888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2633ee-62e3-45c3-9133-ca2a633d882a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:46.818207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 645/1288] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:46.823723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:46.824187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:46.824311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:46.843193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:46.844422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:46.875943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:46.913716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 644/1289] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:47.197416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2633ee-62e3-45c3-9133-ca2a633d882a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:47.303618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2633ee-62e3-45c3-9133-ca2a633d882a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:47.314555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:47.338876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:47.339506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:47.340069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:47.340350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:47.340933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:47.342312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:47.351249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:47.356274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:47.357346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 646/1290] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:55:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:47.358531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:47.460168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669406e8-2de7-4d2e-b648-ac9cc24f5c07 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:47.494974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:47.495576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:47.495854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:47.496155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:47.496781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:47.497079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:47.498434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbea6b5-1aec-4085-8622-5905d7593bb4 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:47.503428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 645/1291] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 10:55:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:47.509623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:47.609370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd0c9e0-fc83-4f7b-afb1-a6919e399135 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:47.642574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:47.643177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:47.643455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:47.643724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] PATH_INFO: `/v3/services` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:47.644384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] RBAC: Adding query filter params (type=placement) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:47.644685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:47.646041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e328d-0758-4648-aa35-0a5f90babf1e None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:47.650401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 647/1292] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 10:55:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:49.569583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6c0c3-56c6-4d2c-b985-a6cd52771c85 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:49.570119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6c0c3-56c6-4d2c-b985-a6cd52771c85 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:49.570119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6c0c3-56c6-4d2c-b985-a6cd52771c85 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:49.571480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 646/1293] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:49.576680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff60b3-3bc1-4fbc-b602-ed75b6c0509d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:49.577067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff60b3-3bc1-4fbc-b602-ed75b6c0509d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:49.577226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff60b3-3bc1-4fbc-b602-ed75b6c0509d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:49.595364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1cff60b3-3bc1-4fbc-b602-ed75b6c0509d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:49.596702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1cff60b3-3bc1-4fbc-b602-ed75b6c0509d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:49.631041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:49.668079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 648/1294] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:49.937132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6c0c3-56c6-4d2c-b985-a6cd52771c85 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:49.969709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:50.051015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6c0c3-56c6-4d2c-b985-a6cd52771c85 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:50.090243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab917418-af5f-4555-96e4-171086fefbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:50.090852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab917418-af5f-4555-96e4-171086fefbca None admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:50.091133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab917418-af5f-4555-96e4-171086fefbca None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:50.091410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab917418-af5f-4555-96e4-171086fefbca None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:50.092119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab917418-af5f-4555-96e4-171086fefbca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:50.093491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab917418-af5f-4555-96e4-171086fefbca None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:50.115499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 647/1295] 10.222.0.20 () {60 vars in 1141 bytes} [Wed Aug 31 10:55:49 2022] POST /identity/v3/services => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 10:55:52.033874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d220d9-061c-471f-8cbf-121bda38427a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.034446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d220d9-061c-471f-8cbf-121bda38427a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.034446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d220d9-061c-471f-8cbf-121bda38427a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.035812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 649/1296] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:52.040994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.041380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.041534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.059366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:52.060576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:52.133149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 648/1297] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:52.397354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d220d9-061c-471f-8cbf-121bda38427a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:52.499959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d220d9-061c-471f-8cbf-121bda38427a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:52.534957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:52.535609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.535889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.536199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.536782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:52.538153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:52.546922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:52.548217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:52.549190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 650/1298] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:55:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:52.554672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:52.573008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:52.654565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac5d7e6-6274-4789-813b-929f39026276 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:52.688152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:52.688761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.689042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.689315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.689947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:52.690251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:52.692185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:52.697261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 649/1299] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 10:55:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:52.703219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:52.734952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:52.806985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767bfc29-176a-4736-baf4-cce3cc94247c None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:52.840668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:52.841259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.841536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.841810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.842506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] RBAC: Adding query filter params (interface=public, service_id=d0c58104fe684499bd997e12c66ee0a5, region_id=RegionOne) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:52.842814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:52.844200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67c31b-3f9d-4ec4-b9ce-2aa7bf4e12ef None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:52.849081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 651/1300] 10.222.0.20 () {56 vars in 1251 bytes} [Wed Aug 31 10:55:52 2022] GET /identity/v3/endpoints?service_id=d0c58104fe684499bd997e12c66ee0a5&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 10:55:52.854774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:52.935197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:52.953969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da848fc8-b513-406a-9d6c-09f39d30f139 None admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:52.990462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050b65ed-b2dd-4522-9ae8-a59ee7d81028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:52.991217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b65ed-b2dd-4522-9ae8-a59ee7d81028 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:52.991621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b65ed-b2dd-4522-9ae8-a59ee7d81028 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:52.991948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050b65ed-b2dd-4522-9ae8-a59ee7d81028 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:52.992755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b65ed-b2dd-4522-9ae8-a59ee7d81028 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:52.994340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050b65ed-b2dd-4522-9ae8-a59ee7d81028 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:53.003845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 650/1301] 10.222.0.20 () {56 vars in 1087 bytes} [Wed Aug 31 10:55:52 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 10:55:54.971965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e5f2e-b163-4af6-91af-8fdce1ecc87a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:54.972559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e5f2e-b163-4af6-91af-8fdce1ecc87a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:54.972652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e5f2e-b163-4af6-91af-8fdce1ecc87a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:54.973948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 652/1302] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:55:54.979411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:54.980488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:54.980488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:54.998080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:55:54.999027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:55:55.065099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 651/1303] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:55:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 10:55:55.336446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e5f2e-b163-4af6-91af-8fdce1ecc87a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:55.433233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:55.441900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e5f2e-b163-4af6-91af-8fdce1ecc87a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:55.475820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:55.476477 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:55.476753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:55.477031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:55.477613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:55.478971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 10:55:55.488274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 10:55:55.489598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application with Lock( Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self._enter() Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application return self.creator() Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application created_value = creator( Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: ERROR keystone.server.flask.application Aug 31 10:55:55.490687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 653/1304] 10.222.0.20 () {56 vars in 1117 bytes} [Wed Aug 31 10:55:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 10:55:55.495637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:55.588375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:55:55.616021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12076c1d-7a39-4188-9ce9-4cc7507c203d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:55.655982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:55.656619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:55.656904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:55.657178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] PATH_INFO: `/v3/services` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:55.657809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 10:55:55.658115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:55.659589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b05421-78ff-4d33-8b1a-3cf68a5d83c1 None admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:55.664889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 652/1305] 10.222.0.20 () {56 vars in 1116 bytes} [Wed Aug 31 10:55:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 10:55:55.670691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:55:55.777162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735eaa3a-e807-4a4e-972e-a028e6ec63d0 None admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:55:55.811456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c8000-8fdd-4c69-9ee0-b107e9a58788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:55:55.812080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8000-8fdd-4c69-9ee0-b107e9a58788 None admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:55:55.812386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8000-8fdd-4c69-9ee0-b107e9a58788 None admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:55:55.812659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8000-8fdd-4c69-9ee0-b107e9a58788 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:55:55.813339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c8000-8fdd-4c69-9ee0-b107e9a58788 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:55:55.814703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159c8000-8fdd-4c69-9ee0-b107e9a58788 None admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:55:55.836094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 654/1306] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 10:55:55 2022] POST /identity/v3/endpoints => generated 331 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 10:56:05.661834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=30 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:56:05.831923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:01.410465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b73d-cb47-4ac4-8f05-e728489f70c3 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:01.410948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b73d-cb47-4ac4-8f05-e728489f70c3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:01.411042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b73d-cb47-4ac4-8f05-e728489f70c3 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:01.417677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd056-cfe5-43ef-acf4-4a1536b858dc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:01.418108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd056-cfe5-43ef-acf4-4a1536b858dc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:01.418230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dd056-cfe5-43ef-acf4-4a1536b858dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:01.420365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 653/1307] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:01.504815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a95dd056-cfe5-43ef-acf4-4a1536b858dc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:01.506338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a95dd056-cfe5-43ef-acf4-4a1536b858dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:01.632912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 655/1308] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:57:01.939654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76512d02-0614-4a76-a36e-35a0e842d081 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:01.940140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76512d02-0614-4a76-a36e-35a0e842d081 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:01.940527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76512d02-0614-4a76-a36e-35a0e842d081 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:01.942115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 654/1309] 10.222.0.20 () {54 vars in 965 bytes} [Wed Aug 31 10:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:01.948921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07290-0b97-4406-8b44-3c9b1d9e7d26 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:01.949345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07290-0b97-4406-8b44-3c9b1d9e7d26 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:01.949759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb07290-0b97-4406-8b44-3c9b1d9e7d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:02.037073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9eb07290-0b97-4406-8b44-3c9b1d9e7d26 None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:02.038518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9eb07290-0b97-4406-8b44-3c9b1d9e7d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:02.060845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:02.124398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 656/1310] 10.222.0.20 () {58 vars in 1062 bytes} [Wed Aug 31 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:02.133255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76512d02-0614-4a76-a36e-35a0e842d081 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:02.135171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-76512d02-0614-4a76-a36e-35a0e842d081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:02.257429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76512d02-0614-4a76-a36e-35a0e842d081 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:02.337678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:02.338456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:02.338832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:02.339202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:02.340919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:02.414005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:02.479116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:02.480733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce4b3d-ecb7-4799-92ea-85e42f87f347 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:02.501847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 655/1311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:04.775259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fb253-9af4-44c1-9076-e17bc2bee8d9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:04.775259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fb253-9af4-44c1-9076-e17bc2bee8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:04.775259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489fb253-9af4-44c1-9076-e17bc2bee8d9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:04.776139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 657/1312] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:04.784077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e971d-e756-4d0a-a84e-208afe007b51 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:04.784077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e971d-e756-4d0a-a84e-208afe007b51 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:04.784077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e971d-e756-4d0a-a84e-208afe007b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:04.867170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ee5e971d-e756-4d0a-a84e-208afe007b51 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:04.868399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ee5e971d-e756-4d0a-a84e-208afe007b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:04.968698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 656/1313] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:05.252843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fb253-9af4-44c1-9076-e17bc2bee8d9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:05.254421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-489fb253-9af4-44c1-9076-e17bc2bee8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:05.360130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489fb253-9af4-44c1-9076-e17bc2bee8d9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:05.438526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:05.441823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:05.442543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:05.443010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:05.443485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:05.445371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:05.552748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:05.554086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff08d9-10d8-402f-ba9b-0f4475730c46 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:05.571841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 658/1314] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:07.790508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71af46b-918b-409c-84c2-6b9d27525a6f None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:07.790954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71af46b-918b-409c-84c2-6b9d27525a6f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:07.791086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71af46b-918b-409c-84c2-6b9d27525a6f None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:07.792503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 657/1315] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:07.798547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa40ca-7db7-4544-a908-cab73877a03e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:07.798984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa40ca-7db7-4544-a908-cab73877a03e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:07.799141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa40ca-7db7-4544-a908-cab73877a03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:07.872749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-49fa40ca-7db7-4544-a908-cab73877a03e None None] MFA Rules not processed for user `940a93c8f7514875b6777c4fd2aaa85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:07.874741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-49fa40ca-7db7-4544-a908-cab73877a03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:07.928685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:07.973289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 659/1316] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:57:08.265752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71af46b-918b-409c-84c2-6b9d27525a6f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:08.267291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c71af46b-918b-409c-84c2-6b9d27525a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:08.277292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:08.374165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71af46b-918b-409c-84c2-6b9d27525a6f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:08.448628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:08.449234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:08.449512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:08.449781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:08.451630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:08.515435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:08.562476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:08.563900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b899998-460c-4003-8eba-801b9aa618a4 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:08.581948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 658/1317] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:11.102153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78501-3c4e-4620-9c5e-6fd6a52dc403 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:11.102716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78501-3c4e-4620-9c5e-6fd6a52dc403 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:11.102716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78501-3c4e-4620-9c5e-6fd6a52dc403 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:11.103981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 660/1318] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:11.109925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1e633-36e3-4c60-b320-0182f24715f1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:11.110554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1e633-36e3-4c60-b320-0182f24715f1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:11.110554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1e633-36e3-4c60-b320-0182f24715f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:11.176099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f2a1e633-36e3-4c60-b320-0182f24715f1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:11.177283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f2a1e633-36e3-4c60-b320-0182f24715f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:11.275733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 659/1319] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:11.564999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78501-3c4e-4620-9c5e-6fd6a52dc403 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:11.566544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-64d78501-3c4e-4620-9c5e-6fd6a52dc403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:11.675526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78501-3c4e-4620-9c5e-6fd6a52dc403 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:11.748675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:11.749276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:11.749552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:11.749820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:11.751419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:11.759970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:11.868066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:11.869413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cefe44-fc52-44d2-835a-f5ca44ee3ac7 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:11.887994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 661/1320] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:14.473036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64475f4-a425-418e-b98e-ed952bfcb1ed None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:14.473626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64475f4-a425-418e-b98e-ed952bfcb1ed None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:14.473626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64475f4-a425-418e-b98e-ed952bfcb1ed None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:14.474926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 660/1321] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:14.480257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51ef39-294a-4854-af39-63e7a77a055a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:14.480779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51ef39-294a-4854-af39-63e7a77a055a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:14.480878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51ef39-294a-4854-af39-63e7a77a055a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:14.552430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ba51ef39-294a-4854-af39-63e7a77a055a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:14.555268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ba51ef39-294a-4854-af39-63e7a77a055a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:14.622626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:14.664487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 662/1322] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:15.030876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2fd85-2646-44b0-91af-59ead2e6946d None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:15.031156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2fd85-2646-44b0-91af-59ead2e6946d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:15.031435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e2fd85-2646-44b0-91af-59ead2e6946d None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:15.032856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 661/1323] 10.222.0.20 () {54 vars in 965 bytes} [Wed Aug 31 10:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:15.038996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e276a-a405-4239-9ee6-73f19befe221 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:15.039296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e276a-a405-4239-9ee6-73f19befe221 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:15.039635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e276a-a405-4239-9ee6-73f19befe221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:15.133335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-559e276a-a405-4239-9ee6-73f19befe221 None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:15.134556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-559e276a-a405-4239-9ee6-73f19befe221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:15.237344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 663/1324] 10.222.0.20 () {58 vars in 1062 bytes} [Wed Aug 31 10:57:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:57:15.246532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2fd85-2646-44b0-91af-59ead2e6946d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:15.248077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d3e2fd85-2646-44b0-91af-59ead2e6946d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:15.274540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:15.372693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e2fd85-2646-44b0-91af-59ead2e6946d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:15.448449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:15.449051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:15.449325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:15.449718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:15.451625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:15.524240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:15.565215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:15.566650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f24e01-70dd-4d14-a9d6-836f3acbef00 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:15.587963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 662/1325] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:17.824034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b533-5738-4a31-87e3-6caf98675cf0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:17.824571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b533-5738-4a31-87e3-6caf98675cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:17.824676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95b533-5738-4a31-87e3-6caf98675cf0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:17.826753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 664/1326] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:17.832086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ecd54-7102-432a-84a4-3f925cc3cca6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:17.832592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ecd54-7102-432a-84a4-3f925cc3cca6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:17.832693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ecd54-7102-432a-84a4-3f925cc3cca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:17.900810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a48ecd54-7102-432a-84a4-3f925cc3cca6 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:17.902125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a48ecd54-7102-432a-84a4-3f925cc3cca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:18.002136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 663/1327] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:18.289589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95b533-5738-4a31-87e3-6caf98675cf0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:18.291131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4e95b533-5738-4a31-87e3-6caf98675cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:18.356074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:18.402081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95b533-5738-4a31-87e3-6caf98675cf0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:18.478635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:18.479248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:18.479527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:18.479801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:18.481452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:18.589903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:18.593176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:18.595113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31493b38-92f3-492a-8fb9-56ca27e27e77 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:18.614050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 665/1328] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:20.611267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c668121-3380-48cf-b0fa-5d03880f57de None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:20.611785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c668121-3380-48cf-b0fa-5d03880f57de None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:20.611785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c668121-3380-48cf-b0fa-5d03880f57de None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:20.613204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 664/1329] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:20.618866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da84e7-381c-4f28-b01d-65ee1a9b3cd2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:20.619394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da84e7-381c-4f28-b01d-65ee1a9b3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:20.619394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da84e7-381c-4f28-b01d-65ee1a9b3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:20.696530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e6da84e7-381c-4f28-b01d-65ee1a9b3cd2 None None] MFA Rules not processed for user `940a93c8f7514875b6777c4fd2aaa85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:20.697772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e6da84e7-381c-4f28-b01d-65ee1a9b3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:20.805362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 666/1330] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:21.097049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c668121-3380-48cf-b0fa-5d03880f57de None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:21.098574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4c668121-3380-48cf-b0fa-5d03880f57de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:21.133080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:21.239639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c668121-3380-48cf-b0fa-5d03880f57de None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:21.337694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:21.338310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:21.338596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:21.339633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:21.340740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:21.436039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:21.470542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:21.471993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12201333-ac5f-4ef9-8be5-2493fc8dbc88 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:21.493383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 665/1331] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:21 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:23.868822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554468bb-c0f5-4b58-8fe5-c7c27eb00119 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:23.869363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554468bb-c0f5-4b58-8fe5-c7c27eb00119 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:23.869363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554468bb-c0f5-4b58-8fe5-c7c27eb00119 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:23.870742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 667/1332] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:23.876552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aeda5-cdd8-42b8-a0fa-7d8fe9efcd67 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:23.877110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aeda5-cdd8-42b8-a0fa-7d8fe9efcd67 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:23.877110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61aeda5-cdd8-42b8-a0fa-7d8fe9efcd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:23.946986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e61aeda5-cdd8-42b8-a0fa-7d8fe9efcd67 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:23.948237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e61aeda5-cdd8-42b8-a0fa-7d8fe9efcd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:24.047326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 666/1333] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:24.332862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554468bb-c0f5-4b58-8fe5-c7c27eb00119 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:24.334390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-554468bb-c0f5-4b58-8fe5-c7c27eb00119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:24.404677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:24.443193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554468bb-c0f5-4b58-8fe5-c7c27eb00119 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:24.519122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:24.519750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:24.520048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:24.520323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:24.521967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:24.636325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:24.637680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a4a71-9339-46cf-8e1c-4d3bb0cf5917 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:24.649202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:24.658605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 668/1334] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:26.636279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b451b6-ae34-49ba-9621-cd5c5b344a86 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:26.636848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b451b6-ae34-49ba-9621-cd5c5b344a86 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:26.636848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b451b6-ae34-49ba-9621-cd5c5b344a86 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:26.638216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 667/1335] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:26.643751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c672e6-7a3a-4342-bce2-5e747fb48f80 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:26.644228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c672e6-7a3a-4342-bce2-5e747fb48f80 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:26.644323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c672e6-7a3a-4342-bce2-5e747fb48f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:26.711255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a5c672e6-7a3a-4342-bce2-5e747fb48f80 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:26.712523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a5c672e6-7a3a-4342-bce2-5e747fb48f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:26.811836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 669/1336] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:27.102669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b451b6-ae34-49ba-9621-cd5c5b344a86 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:27.104458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-99b451b6-ae34-49ba-9621-cd5c5b344a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:27.151579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:27.224390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b451b6-ae34-49ba-9621-cd5c5b344a86 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:27.296924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:27.297558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:27.297838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:27.298111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:27.299724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:27.393895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:27.421033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:27.422536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65578e32-140c-45f7-a44d-4dc7db428594 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:27.440346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 668/1337] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:29.852619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced646da-976e-43e6-bdc7-81d70dd87bb4 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:29.853141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced646da-976e-43e6-bdc7-81d70dd87bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:29.853141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced646da-976e-43e6-bdc7-81d70dd87bb4 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:29.854518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 670/1338] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:29.859830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ad1be-4da9-4ebe-8fc4-b1ea63151741 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:29.860339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ad1be-4da9-4ebe-8fc4-b1ea63151741 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:29.860436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ad1be-4da9-4ebe-8fc4-b1ea63151741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:29.928751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b82ad1be-4da9-4ebe-8fc4-b1ea63151741 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:29.929956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b82ad1be-4da9-4ebe-8fc4-b1ea63151741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:30.037037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 669/1339] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:30.334599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced646da-976e-43e6-bdc7-81d70dd87bb4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:30.336166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ced646da-976e-43e6-bdc7-81d70dd87bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:30.411094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:30.441197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced646da-976e-43e6-bdc7-81d70dd87bb4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:30.515596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:30.516234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:30.516567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:30.516864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:30.518502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:30.628577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:30.629909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363b803-385b-480b-9100-92a45c5539f7 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:30.647362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 671/1340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:34.984826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa3eb7-e9bc-4451-bc6b-bfeb388e6876 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:34.985459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa3eb7-e9bc-4451-bc6b-bfeb388e6876 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:34.985459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa3eb7-e9bc-4451-bc6b-bfeb388e6876 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:34.986761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 670/1341] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:34.993829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094787b-7823-495a-8e68-4ee391bb2b06 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:34.994377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094787b-7823-495a-8e68-4ee391bb2b06 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:34.994494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094787b-7823-495a-8e68-4ee391bb2b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:35.000938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:35.069827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c094787b-7823-495a-8e68-4ee391bb2b06 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:35.071077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c094787b-7823-495a-8e68-4ee391bb2b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:35.199343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 672/1342] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:57:35.515758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa3eb7-e9bc-4451-bc6b-bfeb388e6876 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:35.517352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6cfa3eb7-e9bc-4451-bc6b-bfeb388e6876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:35.576085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:35.660495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa3eb7-e9bc-4451-bc6b-bfeb388e6876 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:35.751424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:35.752075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:35.752337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:35.752610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:35.754236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:35.878479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:35.898203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:35.899961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b860197-6afd-46fc-ac3a-880e093f34d6 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:35.920557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 671/1343] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:38.608375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a6b0f-a3cf-4f3c-bc3e-b746d9b8bd46 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:38.609090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a6b0f-a3cf-4f3c-bc3e-b746d9b8bd46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:38.609090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a6b0f-a3cf-4f3c-bc3e-b746d9b8bd46 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:38.610296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 673/1344] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:38.615819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b639a5-997a-4c55-b416-3219e2fbde10 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:38.616325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b639a5-997a-4c55-b416-3219e2fbde10 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:38.616526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b639a5-997a-4c55-b416-3219e2fbde10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:38.694509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b8b639a5-997a-4c55-b416-3219e2fbde10 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:38.695931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b8b639a5-997a-4c55-b416-3219e2fbde10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:38.816802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 672/1345] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:39.121554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a6b0f-a3cf-4f3c-bc3e-b746d9b8bd46 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:39.124454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-614a6b0f-a3cf-4f3c-bc3e-b746d9b8bd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:39.223541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:39.244100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a6b0f-a3cf-4f3c-bc3e-b746d9b8bd46 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:39.334933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:39.335562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:39.335849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:39.336157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:39.337894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:39.472931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:39.474451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3a52d-0d57-43c7-94a4-7853663be793 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:39.495743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 674/1346] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:43.692153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905ab9-f725-45bf-be8e-448fcd2df629 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:43.692846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905ab9-f725-45bf-be8e-448fcd2df629 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:43.692846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06905ab9-f725-45bf-be8e-448fcd2df629 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:43.694132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 673/1347] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:43.701512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ca2c-fbfd-4024-8a6a-2afa5f30e7c8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:43.702123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ca2c-fbfd-4024-8a6a-2afa5f30e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:43.702413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677ca2c-fbfd-4024-8a6a-2afa5f30e7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:43.720752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:43.781060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9677ca2c-fbfd-4024-8a6a-2afa5f30e7c8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:43.782436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9677ca2c-fbfd-4024-8a6a-2afa5f30e7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:43.898039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 675/1348] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:44.202096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905ab9-f725-45bf-be8e-448fcd2df629 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:57:44.203644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-06905ab9-f725-45bf-be8e-448fcd2df629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:44.277489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:44.334242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06905ab9-f725-45bf-be8e-448fcd2df629 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:57:44.421877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:57:44.422592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:44.422928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:44.423384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:44.425281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:44.580748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:44.584962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:57:44.586897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879ba381-015e-4eba-8bb2-6c3e81d022f0 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:57:44.610449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 674/1349] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:57:50.256215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b1245-ee51-4fba-a305-f7581b349ff5 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:50.256215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b1245-ee51-4fba-a305-f7581b349ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:50.256810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b1245-ee51-4fba-a305-f7581b349ff5 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:50.258073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 676/1350] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 10:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:50.266100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a161030-435b-4b4b-b6d1-ff1da375b3a1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:50.266609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a161030-435b-4b4b-b6d1-ff1da375b3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:50.266716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a161030-435b-4b4b-b6d1-ff1da375b3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:50.423284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0a161030-435b-4b4b-b6d1-ff1da375b3a1 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:50.424554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0a161030-435b-4b4b-b6d1-ff1da375b3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:50.571978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 675/1351] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 10:57:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:53.896892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=59 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:55.613862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14929568-16df-4981-a42f-702a30037672 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:55.614498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14929568-16df-4981-a42f-702a30037672 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:55.614498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14929568-16df-4981-a42f-702a30037672 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:55.616091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 677/1352] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 10:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:55.622145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729f0f3-5125-45d9-9ddb-27d453354dd5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:55.622730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729f0f3-5125-45d9-9ddb-27d453354dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:55.622730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729f0f3-5125-45d9-9ddb-27d453354dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:55.684889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:55.713941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1729f0f3-5125-45d9-9ddb-27d453354dd5 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:55.717319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1729f0f3-5125-45d9-9ddb-27d453354dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:55.926814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 676/1353] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 10:57:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:57:57.400231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc87f90-e627-4262-9b1f-007e618a2b71 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:57.401026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc87f90-e627-4262-9b1f-007e618a2b71 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:57.401431 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc87f90-e627-4262-9b1f-007e618a2b71 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:57.403249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 678/1354] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 10:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:57:57.408864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a4dca-0cef-4fe0-8435-74eba59e7625 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:57:57.409533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a4dca-0cef-4fe0-8435-74eba59e7625 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:57:57.409533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a4dca-0cef-4fe0-8435-74eba59e7625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:57:57.495223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-259a4dca-0cef-4fe0-8435-74eba59e7625 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:57:57.497101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-259a4dca-0cef-4fe0-8435-74eba59e7625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:57:57.543612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:57:57.620192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 677/1355] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 10:57:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:01.109264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bdda7-7642-4ab7-a3f6-d38d05fe55cd None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:01.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bdda7-7642-4ab7-a3f6-d38d05fe55cd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:01.109906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894bdda7-7642-4ab7-a3f6-d38d05fe55cd None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:01.111179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 679/1356] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:01.117149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c2c1-6be4-4721-ac2a-6aca8ac850d3 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:01.117628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c2c1-6be4-4721-ac2a-6aca8ac850d3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:01.117828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b2c2c1-6be4-4721-ac2a-6aca8ac850d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:01.206661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c9b2c2c1-6be4-4721-ac2a-6aca8ac850d3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:01.207837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c9b2c2c1-6be4-4721-ac2a-6aca8ac850d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:01.338536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 678/1357] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:01.684418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746fda-f403-4027-9e73-f48633afd728 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:01.685002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746fda-f403-4027-9e73-f48633afd728 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:01.685423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8746fda-f403-4027-9e73-f48633afd728 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:01.688218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 680/1358] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 10:58:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:01.703824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ad14-925a-464d-957b-951f4fa7e3b5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:01.704616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ad14-925a-464d-957b-951f4fa7e3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:01.705681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029ad14-925a-464d-957b-951f4fa7e3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:01.729134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:01.828174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7029ad14-925a-464d-957b-951f4fa7e3b5 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:01.829568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7029ad14-925a-464d-957b-951f4fa7e3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:01.984484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 679/1359] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 10:58:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:01.998949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746fda-f403-4027-9e73-f48633afd728 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:02.000778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8746fda-f403-4027-9e73-f48633afd728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:02.204273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8746fda-f403-4027-9e73-f48633afd728 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:02.311683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:02.381796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:02.382421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:02.382833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:02.383274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:02.385727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:02.550421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:02.553314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fed49f-2002-4985-9e2f-2a25255ea1d9 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:02.588682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 681/1360] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:58:05.134300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d1a2-8612-4e73-a745-a489ac8bb86a None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:05.135220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d1a2-8612-4e73-a745-a489ac8bb86a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:05.135220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9d1a2-8612-4e73-a745-a489ac8bb86a None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:05.137338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 680/1361] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:58:05.143229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ed14e-a1b4-4551-a4d1-0783080f415b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:05.144156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ed14e-a1b4-4551-a4d1-0783080f415b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:05.144456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ed14e-a1b4-4551-a4d1-0783080f415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:05.252828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5d5ed14e-a1b4-4551-a4d1-0783080f415b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:05.254139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d5ed14e-a1b4-4551-a4d1-0783080f415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:05.297979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:05.371955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 682/1362] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:58:05.668567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9d1a2-8612-4e73-a745-a489ac8bb86a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:05.670090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-66f9d1a2-8612-4e73-a745-a489ac8bb86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:05.754889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:05.925575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9d1a2-8612-4e73-a745-a489ac8bb86a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:06.128888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:06.129382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:06.129669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:06.129944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:06.131614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:06.160441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:06.287882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:06.290767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778608d1-6751-419e-992c-433ee9614e2b service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:06.319955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 681/1363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 652 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:08.716423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49944581-db72-4010-914b-11fd950c7510 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:08.716979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49944581-db72-4010-914b-11fd950c7510 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:08.717390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49944581-db72-4010-914b-11fd950c7510 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:08.720101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 683/1364] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:08.726316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50869-cb2a-471a-9ee6-94d9011aa863 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:08.727071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50869-cb2a-471a-9ee6-94d9011aa863 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:08.727071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50869-cb2a-471a-9ee6-94d9011aa863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:08.834621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a2c50869-cb2a-471a-9ee6-94d9011aa863 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:08.835876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a2c50869-cb2a-471a-9ee6-94d9011aa863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:08.965018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:08.972191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 682/1365] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:58:09.433832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e7c1-9ccd-42f5-a623-66b07f9074c9 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:09.434150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e7c1-9ccd-42f5-a623-66b07f9074c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:09.434886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489e7c1-9ccd-42f5-a623-66b07f9074c9 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:09.436702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 684/1366] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 10:58:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:09.442473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0cec6-a783-4512-9a95-8fa0ab3b7d38 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:09.442826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0cec6-a783-4512-9a95-8fa0ab3b7d38 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:09.443112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0cec6-a783-4512-9a95-8fa0ab3b7d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:09.539021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-17b0cec6-a783-4512-9a95-8fa0ab3b7d38 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:09.540320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-17b0cec6-a783-4512-9a95-8fa0ab3b7d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:09.685751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 683/1367] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 10:58:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:09.693918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489e7c1-9ccd-42f5-a623-66b07f9074c9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:09.695652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2489e7c1-9ccd-42f5-a623-66b07f9074c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:09.872603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489e7c1-9ccd-42f5-a623-66b07f9074c9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:09.896290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:10.007984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:10.008542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:10.008826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:10.009101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:10.011076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:10.117515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:10.118906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72a949-d153-4ae6-9b9b-8684aefde741 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:10.141615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 685/1368] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:10.188837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93abd-3d25-4150-a03d-a8d0734554a4 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:10.189415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93abd-3d25-4150-a03d-a8d0734554a4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:10.189415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c93abd-3d25-4150-a03d-a8d0734554a4 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:10.191182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 684/1369] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:10.199076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faaf51-0938-4947-afa2-d51ce5177657 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:10.199076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faaf51-0938-4947-afa2-d51ce5177657 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:10.199076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9faaf51-0938-4947-afa2-d51ce5177657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:10.236764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:10.281344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f9faaf51-0938-4947-afa2-d51ce5177657 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:10.282659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9faaf51-0938-4947-afa2-d51ce5177657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:10.427193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 686/1370] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:11.019550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae9075-9f24-4990-8ded-bd5ca89a8577 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:11.019550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae9075-9f24-4990-8ded-bd5ca89a8577 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:11.019550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae9075-9f24-4990-8ded-bd5ca89a8577 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:11.020834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 685/1371] 10.222.0.20 () {54 vars in 964 bytes} [Wed Aug 31 10:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:11.025071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f5e2-dd2d-4a1c-9174-6712f0e18c89 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:11.025622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f5e2-dd2d-4a1c-9174-6712f0e18c89 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:11.025622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027f5e2-dd2d-4a1c-9174-6712f0e18c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:11.109915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2027f5e2-dd2d-4a1c-9174-6712f0e18c89 None None] MFA Rules not processed for user `e1aaf56b3e654c0bb65d0c804fd797e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:11.111159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2027f5e2-dd2d-4a1c-9174-6712f0e18c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:11.172484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:11.220185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 687/1372] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Aug 31 10:58:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:11.232917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae9075-9f24-4990-8ded-bd5ca89a8577 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:11.234828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a0ae9075-9f24-4990-8ded-bd5ca89a8577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:11.312863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:11.396709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ae9075-9f24-4990-8ded-bd5ca89a8577 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:11.480328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:11.480947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:11.481229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:11.481504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:11.483765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:11.599458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:11.617826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:11.619208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a28e6ba-bc5d-48a8-847f-0f1101d845ce service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:11.639394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 686/1373] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:12.191965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0cbf3-d4e8-47cd-8a76-9d728da8be85 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:12.192562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0cbf3-d4e8-47cd-8a76-9d728da8be85 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:12.192562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b0cbf3-d4e8-47cd-8a76-9d728da8be85 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:12.193883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 688/1374] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:12.200057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899c04-e458-4ca0-b39d-20818794fc03 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:12.200341 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899c04-e458-4ca0-b39d-20818794fc03 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:12.200612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32899c04-e458-4ca0-b39d-20818794fc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:12.271894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-32899c04-e458-4ca0-b39d-20818794fc03 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:12.273266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-32899c04-e458-4ca0-b39d-20818794fc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:12.386171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 687/1375] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:12.694574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0cbf3-d4e8-47cd-8a76-9d728da8be85 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:12.696144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-10b0cbf3-d4e8-47cd-8a76-9d728da8be85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:12.848575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b0cbf3-d4e8-47cd-8a76-9d728da8be85 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:12.903930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:12.972437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:12.973048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:12.973324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:12.973600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:12.975220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:13.075487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:13.077164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da9b25a-835e-4d39-a667-b2ae8edb570c service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:13.101255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 689/1376] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:15.270430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d37a-83fb-4c9a-a274-744fdb95bf43 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:15.270956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d37a-83fb-4c9a-a274-744fdb95bf43 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:15.271045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532d37a-83fb-4c9a-a274-744fdb95bf43 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:15.272489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 688/1377] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:58:15.278441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3c4d-09c0-45d2-a533-84eef50690c8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:15.279339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3c4d-09c0-45d2-a533-84eef50690c8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:15.279339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf3c4d-09c0-45d2-a533-84eef50690c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:15.379351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c8bf3c4d-09c0-45d2-a533-84eef50690c8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:15.380955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c8bf3c4d-09c0-45d2-a533-84eef50690c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:15.389812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:15.528230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 690/1378] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:15.853082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532d37a-83fb-4c9a-a274-744fdb95bf43 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:15.855317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f532d37a-83fb-4c9a-a274-744fdb95bf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:15.935047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:16.056681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532d37a-83fb-4c9a-a274-744fdb95bf43 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:16.204667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:16.205555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:16.205848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:16.206108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:16.207750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:16.233259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:16.327746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:16.330119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc83c70b-f076-43d1-940a-c64817fb1cc0 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:16.351285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 689/1379] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:18.377536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049a44c-50f0-4e68-ad29-487e23ddcfc3 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:18.378112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049a44c-50f0-4e68-ad29-487e23ddcfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:18.378112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e049a44c-50f0-4e68-ad29-487e23ddcfc3 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:18.379786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 691/1380] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:58:18.388438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330fa15-2dd7-4dd7-9acc-16b0baeeb076 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:18.389007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330fa15-2dd7-4dd7-9acc-16b0baeeb076 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:18.389116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330fa15-2dd7-4dd7-9acc-16b0baeeb076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:18.468942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2330fa15-2dd7-4dd7-9acc-16b0baeeb076 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:18.472499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2330fa15-2dd7-4dd7-9acc-16b0baeeb076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:18.586538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:18.597747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 690/1381] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:18.898122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049a44c-50f0-4e68-ad29-487e23ddcfc3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:18.899842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e049a44c-50f0-4e68-ad29-487e23ddcfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:19.052775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:19.065276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e049a44c-50f0-4e68-ad29-487e23ddcfc3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:19.190559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:19.191316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:19.191698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:19.192107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:19.194134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:19.289985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:19.292349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456c801-8fb3-48a3-b09c-f9819577e464 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:19.303448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:19.320197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 692/1382] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:21.805061 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c491c-4de3-4ec9-ac69-6e9858b296ee None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:21.805617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c491c-4de3-4ec9-ac69-6e9858b296ee None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:21.805753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c491c-4de3-4ec9-ac69-6e9858b296ee None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:21.808250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 691/1383] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:21.813840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997ed9-b231-41ad-b2bc-9e403cc004d4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:21.814334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997ed9-b231-41ad-b2bc-9e403cc004d4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:21.814449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17997ed9-b231-41ad-b2bc-9e403cc004d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:21.890693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-17997ed9-b231-41ad-b2bc-9e403cc004d4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:21.891946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-17997ed9-b231-41ad-b2bc-9e403cc004d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:22.006382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 693/1384] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:22.295785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c491c-4de3-4ec9-ac69-6e9858b296ee None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:22.298901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-905c491c-4de3-4ec9-ac69-6e9858b296ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:22.470080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c491c-4de3-4ec9-ac69-6e9858b296ee None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:22.545098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:22.608646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:22.609273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:22.609548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:22.609815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:22.611522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:22.709547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:22.711154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d54190-3202-4ad5-8394-aa3d4c5a81e8 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:22.730758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 692/1385] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:24.735108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42065fe0-dc2c-49a0-80d9-70b259522669 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:24.736090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42065fe0-dc2c-49a0-80d9-70b259522669 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:24.736090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42065fe0-dc2c-49a0-80d9-70b259522669 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:24.737164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 694/1386] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:24.743097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ccef-5f53-43d6-a166-e4b67ddadb97 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:24.743624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ccef-5f53-43d6-a166-e4b67ddadb97 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:24.743731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44ccef-5f53-43d6-a166-e4b67ddadb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:24.832669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8c44ccef-5f53-43d6-a166-e4b67ddadb97 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:24.833925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8c44ccef-5f53-43d6-a166-e4b67ddadb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:24.877273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:24.957853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 693/1387] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:58:25.261134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42065fe0-dc2c-49a0-80d9-70b259522669 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:25.264865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42065fe0-dc2c-49a0-80d9-70b259522669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:25.382070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:25.483750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42065fe0-dc2c-49a0-80d9-70b259522669 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:25.608102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:25.608617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:25.608894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:25.609164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:25.610791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:25.656553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:25.723238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:25.724933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd786f-e3dc-4c31-9cdf-9a235ac4a12a service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:25.746520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 695/1388] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:27.420249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b48b-1176-4dfc-92b3-7ba99b31a2a6 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:27.420822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b48b-1176-4dfc-92b3-7ba99b31a2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:27.420822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8b48b-1176-4dfc-92b3-7ba99b31a2a6 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:27.422194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 694/1389] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:27.427679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577fa5-70de-4454-bcfe-ec2c2c0d56f1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:27.428192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577fa5-70de-4454-bcfe-ec2c2c0d56f1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:27.428292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577fa5-70de-4454-bcfe-ec2c2c0d56f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:27.504203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0f577fa5-70de-4454-bcfe-ec2c2c0d56f1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:27.505422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0f577fa5-70de-4454-bcfe-ec2c2c0d56f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:27.614192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 696/1390] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:27.813625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cce6e-ec26-472b-bcb1-9e9edc6029d2 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:27.813909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cce6e-ec26-472b-bcb1-9e9edc6029d2 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:27.814186 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924cce6e-ec26-472b-bcb1-9e9edc6029d2 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:27.815594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 695/1391] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:27.821447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437344-a915-4063-b827-94a44db87127 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:27.821735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437344-a915-4063-b827-94a44db87127 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:27.822034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c437344-a915-4063-b827-94a44db87127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:27.839182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:27.891931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4c437344-a915-4063-b827-94a44db87127 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:27.893178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4c437344-a915-4063-b827-94a44db87127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:27.926290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c00a8-d592-4708-bd62-a2023e200e73 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:27.926669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c00a8-d592-4708-bd62-a2023e200e73 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:27.927034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c00a8-d592-4708-bd62-a2023e200e73 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:27.928522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 696/1392] 10.222.0.20 () {54 vars in 967 bytes} [Wed Aug 31 10:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:58:27.934250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:27.934533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:27.934812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:28.000432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 697/1393] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:28.019555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] MFA Rules not processed for user `72251fea0df54206b554236c14296a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:28.020883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:28.102393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 697/1394] 10.222.0.20 () {58 vars in 1064 bytes} [Wed Aug 31 10:58:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:58:28.110607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c437344-a915-4063-b827-94a44db87127 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:28.111808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4c437344-a915-4063-b827-94a44db87127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:28.221056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c437344-a915-4063-b827-94a44db87127 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:28.229304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:28.290019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:28.292031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:28.298336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:28.298949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:28.299226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:28.299495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:28.301147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:28.325815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:28.418724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:28.420214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8ec27-f8b0-40a8-8e91-011475577500 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:28.428475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e3de1-577d-4020-870b-d02cac677d5c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:28.440333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 698/1395] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:28.550922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:28.551528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:28.551811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:28.552118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:28.553715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:28.572772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:28.648920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:28.653427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa9070-6635-47b2-a469-56b3bd01a072 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:28.679372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 698/1396] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:58:29.202263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c3eb1-aad5-4b37-98b3-ba4a7b2ab7cb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:29.202653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c3eb1-aad5-4b37-98b3-ba4a7b2ab7cb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:29.203005 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c3eb1-aad5-4b37-98b3-ba4a7b2ab7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:29.250499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:29.273682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2d4c3eb1-aad5-4b37-98b3-ba4a7b2ab7cb None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:29.274896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2d4c3eb1-aad5-4b37-98b3-ba4a7b2ab7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:29.363912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 699/1397] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:29.415786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba04640-7897-4bbf-b2a7-8f0dbf153ce4 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:29.416870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba04640-7897-4bbf-b2a7-8f0dbf153ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:29.417399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba04640-7897-4bbf-b2a7-8f0dbf153ce4 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:29.418858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 699/1398] 10.222.0.20 () {54 vars in 943 bytes} [Wed Aug 31 10:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:29.425294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b10146-b194-4cec-9175-b70843ab1b70 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:29.425649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b10146-b194-4cec-9175-b70843ab1b70 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:29.425994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b10146-b194-4cec-9175-b70843ab1b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:29.510641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e5b10146-b194-4cec-9175-b70843ab1b70 None None] MFA Rules not processed for user `10831ea9480d488ab2a81bece14ff090`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:29.512634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e5b10146-b194-4cec-9175-b70843ab1b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:29.581771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:29.635206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 700/1399] 10.222.0.20 () {58 vars in 1040 bytes} [Wed Aug 31 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:29.645620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba04640-7897-4bbf-b2a7-8f0dbf153ce4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:29.647304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-aba04640-7897-4bbf-b2a7-8f0dbf153ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:29.811252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba04640-7897-4bbf-b2a7-8f0dbf153ce4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:29.820667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:29.926659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:29.927270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:29.927549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:29.927846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:29.929593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:30.021260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:30.022885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0ba6e-1068-4c1f-b9f9-b513609872b1 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:30.032300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 700/1400] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 10:58:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 10:58:30.744949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ec3d5-9025-4eb0-9ddd-296675f20edc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:30.745501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ec3d5-9025-4eb0-9ddd-296675f20edc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:30.745501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ec3d5-9025-4eb0-9ddd-296675f20edc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:30.746888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 701/1401] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:30.752385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3ead-e242-4896-b81a-c073b5994efd None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:30.752852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3ead-e242-4896-b81a-c073b5994efd None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:30.753239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3ead-e242-4896-b81a-c073b5994efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:30.816984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:30.839646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0f0c3ead-e242-4896-b81a-c073b5994efd None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:30.841191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0f0c3ead-e242-4896-b81a-c073b5994efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:30.970098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 701/1402] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:31.254800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ec3d5-9025-4eb0-9ddd-296675f20edc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:31.256660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ff9ec3d5-9025-4eb0-9ddd-296675f20edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:31.428122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ec3d5-9025-4eb0-9ddd-296675f20edc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:31.478332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:31.560382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:31.561242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:31.561696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:31.562144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:31.563790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:31.675336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:31.677105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5e722-1043-49a9-ac0e-20957b689184 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:31.703664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 702/1403] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:33.742037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf993a-812f-4601-8e72-d537f82aa2ed None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:33.742637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf993a-812f-4601-8e72-d537f82aa2ed None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:33.742637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf993a-812f-4601-8e72-d537f82aa2ed None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:33.744557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 702/1404] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:33.749740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315b2f6-283e-42fa-b1ba-fa9f6f85eddd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:33.750180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315b2f6-283e-42fa-b1ba-fa9f6f85eddd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:33.750363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315b2f6-283e-42fa-b1ba-fa9f6f85eddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:33.815440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:33.827002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f315b2f6-283e-42fa-b1ba-fa9f6f85eddd None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:33.828281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f315b2f6-283e-42fa-b1ba-fa9f6f85eddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:33.937743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 703/1405] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:34.233703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf993a-812f-4601-8e72-d537f82aa2ed None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:34.236899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bdaf993a-812f-4601-8e72-d537f82aa2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:34.370888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:34.411065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf993a-812f-4601-8e72-d537f82aa2ed None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:34.545626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:34.546148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:34.546451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:34.546726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:34.548397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:34.638471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:34.652132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:34.653864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18516a59-df6d-4fb7-9451-ce14515ffc78 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:34.674054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 703/1406] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:36.757823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de6b66-b33b-4753-b396-6f491bab1fd0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:36.758386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de6b66-b33b-4753-b396-6f491bab1fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:36.758386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de6b66-b33b-4753-b396-6f491bab1fd0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:36.761494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 704/1407] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 10:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:36.765475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33d51-84bf-448e-a11e-5bb37d8147de None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:36.765918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33d51-84bf-448e-a11e-5bb37d8147de None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:36.766016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d33d51-84bf-448e-a11e-5bb37d8147de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:36.847829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-16d33d51-84bf-448e-a11e-5bb37d8147de None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:36.849119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-16d33d51-84bf-448e-a11e-5bb37d8147de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:37.002704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 704/1408] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 10:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:37.294688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de6b66-b33b-4753-b396-6f491bab1fd0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:37.296281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c5de6b66-b33b-4753-b396-6f491bab1fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:37.430209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:37.463702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de6b66-b33b-4753-b396-6f491bab1fd0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:37.588956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:37.589904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:37.590186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:37.590456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:37.592143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:37.687625 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:37.691497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:37.693166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c49d6-a70a-4ffe-9400-825de1e50862 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:37.715658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 705/1409] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:46.993656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=74 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:47.713395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:53.058451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1758d-123d-491b-b856-e76fcdf17307 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:53.058990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1758d-123d-491b-b856-e76fcdf17307 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:53.058990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1758d-123d-491b-b856-e76fcdf17307 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:53.060368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 705/1410] 10.222.0.20 () {54 vars in 918 bytes} [Wed Aug 31 10:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:53.066542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc9fe2-88a7-443e-81f4-67c4815a12f3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:53.067134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc9fe2-88a7-443e-81f4-67c4815a12f3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:53.067134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc9fe2-88a7-443e-81f4-67c4815a12f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:53.155147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eddc9fe2-88a7-443e-81f4-67c4815a12f3 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:53.156492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eddc9fe2-88a7-443e-81f4-67c4815a12f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:53.266323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 706/1411] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 10:58:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:53.281042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea1c74-de00-4457-859a-9dba7c9cc8ca None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:53.281326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea1c74-de00-4457-859a-9dba7c9cc8ca None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:53.281598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea1c74-de00-4457-859a-9dba7c9cc8ca None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:53.282948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 706/1412] 10.222.0.20 () {54 vars in 918 bytes} [Wed Aug 31 10:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:53.287736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ba659-7d8d-4fe4-883d-47d3a9c3f6a4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:53.288039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ba659-7d8d-4fe4-883d-47d3a9c3f6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:53.288330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525ba659-7d8d-4fe4-883d-47d3a9c3f6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:53.373721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:53.376960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-525ba659-7d8d-4fe4-883d-47d3a9c3f6a4 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:53.378832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-525ba659-7d8d-4fe4-883d-47d3a9c3f6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:53.500819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 707/1413] 10.222.0.20 () {58 vars in 1015 bytes} [Wed Aug 31 10:58:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:55.975359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41b455-23fe-4c43-80e9-0619df4f368e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:55.975994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41b455-23fe-4c43-80e9-0619df4f368e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:55.975994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e41b455-23fe-4c43-80e9-0619df4f368e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:55.978563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:55.984367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21e24-faf5-4053-8251-818f4062dee2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:55.984894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21e24-faf5-4053-8251-818f4062dee2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:55.984894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f21e24-faf5-4053-8251-818f4062dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:56.057703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f7f21e24-faf5-4053-8251-818f4062dee2 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:56.058891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f7f21e24-faf5-4053-8251-818f4062dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:56.119018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:56.163300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:56.454099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41b455-23fe-4c43-80e9-0619df4f368e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:56.455757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2e41b455-23fe-4c43-80e9-0619df4f368e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:56.613858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e41b455-23fe-4c43-80e9-0619df4f368e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:56.689370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:56.733352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:56.733962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:56.734240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:56.734514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:56.736154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:56.833678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:56.835292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62070cc5-159c-4db7-8574-31edec5ffae8 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:56.854449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 708/1416] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:57.785716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b26fba-66a8-4d53-9575-ec929c11ede1 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:57.786397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b26fba-66a8-4d53-9575-ec929c11ede1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:57.786909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b26fba-66a8-4d53-9575-ec929c11ede1 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:57.788861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:58:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:57.794300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b437d-6263-4bb4-980c-7af1d91a6302 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:57.794795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b437d-6263-4bb4-980c-7af1d91a6302 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:57.794938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1b437d-6263-4bb4-980c-7af1d91a6302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:57.922285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-dd1b437d-6263-4bb4-980c-7af1d91a6302 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:57.924364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dd1b437d-6263-4bb4-980c-7af1d91a6302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:57.973541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:58.052966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:58:58.212759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b26fba-66a8-4d53-9575-ec929c11ede1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:58:58.214422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d2b26fba-66a8-4d53-9575-ec929c11ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:58.410809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b26fba-66a8-4d53-9575-ec929c11ede1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:58:58.454591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:58.537209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:58:58.538168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:58.538689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:58.539712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:58.542548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:58.678951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:58:58.681039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabec6d9-b56c-4a00-8a56-37287af2902c service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:58:58.701653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 710/1419] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:58:59.628762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2046ac-06de-495c-a3b0-fefecdde89f0 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:59.629610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2046ac-06de-495c-a3b0-fefecdde89f0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:59.629610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2046ac-06de-495c-a3b0-fefecdde89f0 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:59.630716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:58:59.639243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e79383-58c8-40e4-88c7-6f64ebc33750 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:58:59.639805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e79383-58c8-40e4-88c7-6f64ebc33750 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:58:59.639805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e79383-58c8-40e4-88c7-6f64ebc33750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:58:59.700902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:58:59.727555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-90e79383-58c8-40e4-88c7-6f64ebc33750 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:58:59.728872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-90e79383-58c8-40e4-88c7-6f64ebc33750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:58:59.850664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:00.024115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2046ac-06de-495c-a3b0-fefecdde89f0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:00.025914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6f2046ac-06de-495c-a3b0-fefecdde89f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:00.193733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2046ac-06de-495c-a3b0-fefecdde89f0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:00.208038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:00.328834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:00.329445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:00.329753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:00.330031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:00.331683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:00.435075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:00.436729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c543fc-6a6d-487f-b743-e1fdf6c8ce10 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:00.457044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 711/1422] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:01.375862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe94e29-6520-4c1d-9943-67428ad6f7f2 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:01.376492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe94e29-6520-4c1d-9943-67428ad6f7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:01.376492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe94e29-6520-4c1d-9943-67428ad6f7f2 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:01.377817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:01.383198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19273d-68dc-417a-a0a2-229f0d7e0b83 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:01.383723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19273d-68dc-417a-a0a2-229f0d7e0b83 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:01.383839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19273d-68dc-417a-a0a2-229f0d7e0b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:01.405257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:01.494308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-cf19273d-68dc-417a-a0a2-229f0d7e0b83 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:01.496212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cf19273d-68dc-417a-a0a2-229f0d7e0b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:01.657588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:59:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:01.819447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe94e29-6520-4c1d-9943-67428ad6f7f2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:01.822188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1fe94e29-6520-4c1d-9943-67428ad6f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:01.948784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:01.989883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe94e29-6520-4c1d-9943-67428ad6f7f2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:02.123551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:02.124369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:02.124745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:02.125116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:02.127157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:02.254758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:02.282504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:02.285580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942eb84b-af5a-4fb7-abd1-07297c32316f service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:02.312180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 713/1425] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:03.294982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b115a65-e68f-4a53-acf6-eb9e4faca0cf None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:03.295902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b115a65-e68f-4a53-acf6-eb9e4faca0cf None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:03.295902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b115a65-e68f-4a53-acf6-eb9e4faca0cf None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:03.297458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:59:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:03.303354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474536d9-a8bc-49dd-a399-fbd54cc17d37 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:03.303877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474536d9-a8bc-49dd-a399-fbd54cc17d37 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:03.304025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474536d9-a8bc-49dd-a399-fbd54cc17d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:03.394437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-474536d9-a8bc-49dd-a399-fbd54cc17d37 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:03.395848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-474536d9-a8bc-49dd-a399-fbd54cc17d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:03.534636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:03.688924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b115a65-e68f-4a53-acf6-eb9e4faca0cf None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:03.690467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9b115a65-e68f-4a53-acf6-eb9e4faca0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:03.782859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:03.853237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b115a65-e68f-4a53-acf6-eb9e4faca0cf None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:03.974494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:03.974980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:03.975258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:03.975525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:03.977719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:04.036070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:04.082985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:04.084707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636348b-8219-402f-8bf1-29588bfdeb4c service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:04.104886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 714/1428] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:04.958377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa0d72-67a8-4788-b35f-cb30d044e014 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:04.959057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa0d72-67a8-4788-b35f-cb30d044e014 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:04.959057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa0d72-67a8-4788-b35f-cb30d044e014 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:04.960508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:04.966728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900c48f-7eb7-4a1f-ace4-28170a1d6b7d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:04.967328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900c48f-7eb7-4a1f-ace4-28170a1d6b7d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:04.967328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900c48f-7eb7-4a1f-ace4-28170a1d6b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:05.059189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8900c48f-7eb7-4a1f-ace4-28170a1d6b7d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:05.060516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8900c48f-7eb7-4a1f-ace4-28170a1d6b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:05.169052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:59:05.323051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa0d72-67a8-4788-b35f-cb30d044e014 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:05.324613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4daa0d72-67a8-4788-b35f-cb30d044e014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:05.379023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:05.489199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa0d72-67a8-4788-b35f-cb30d044e014 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:05.617208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:05.617805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:05.618083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:05.618355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:05.620100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:05.627809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:05.721446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:05.723699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5f61d3-944e-48ce-9a2e-bd3ccd869a62 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:05.748600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 716/1431] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:06.581712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217ebd-e24d-421f-9856-d723160a0a41 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:06.582353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217ebd-e24d-421f-9856-d723160a0a41 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:06.582353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e217ebd-e24d-421f-9856-d723160a0a41 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:06.583785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:06.590142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f34a23-6a23-4d9e-aae8-090a4969b413 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:06.590702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f34a23-6a23-4d9e-aae8-090a4969b413 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:06.590843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f34a23-6a23-4d9e-aae8-090a4969b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:06.675034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-26f34a23-6a23-4d9e-aae8-090a4969b413 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:06.676285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-26f34a23-6a23-4d9e-aae8-090a4969b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:06.751979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:06.787547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:59:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:06.946651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e217ebd-e24d-421f-9856-d723160a0a41 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:06.948312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9e217ebd-e24d-421f-9856-d723160a0a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:06.969369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:07.131129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e217ebd-e24d-421f-9856-d723160a0a41 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:07.231137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:07.262915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:07.263534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:07.263818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:07.264126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:07.265818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:07.370108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:07.371964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41795c3c-bfe6-4cd5-90d9-cc17b2a35837 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:07.395743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 717/1434] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:08.270192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9ac6-d7b2-412c-9b38-b4a974616188 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:08.271060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9ac6-d7b2-412c-9b38-b4a974616188 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:08.271060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba9ac6-d7b2-412c-9b38-b4a974616188 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:08.273386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:59:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:59:08.279141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efa2ce-0736-45fc-b6c1-a436d72cd9d7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:08.279833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efa2ce-0736-45fc-b6c1-a436d72cd9d7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:08.279833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efa2ce-0736-45fc-b6c1-a436d72cd9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:08.374863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f3efa2ce-0736-45fc-b6c1-a436d72cd9d7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:08.376323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f3efa2ce-0736-45fc-b6c1-a436d72cd9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:08.427432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:08.492673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:59:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:08.655928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba9ac6-d7b2-412c-9b38-b4a974616188 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:08.657495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bcba9ac6-d7b2-412c-9b38-b4a974616188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:08.832894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba9ac6-d7b2-412c-9b38-b4a974616188 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:08.893705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:08.962244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:08.962761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:08.963042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:08.963313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:08.964997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:09.065289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:09.066929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14ab68-ba4f-4677-9a33-3b7aa3eea464 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:09.097599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 719/1437] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:09.993733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceb59b-5616-48b0-b04a-27406bb72f6a None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:09.994602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceb59b-5616-48b0-b04a-27406bb72f6a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:09.994773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ceb59b-5616-48b0-b04a-27406bb72f6a None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:09.996390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 10:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:59:10.002195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5d19-5b1c-426a-ad45-ccb952a5c7c1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:10.002762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5d19-5b1c-426a-ad45-ccb952a5c7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:10.002762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5d19-5b1c-426a-ad45-ccb952a5c7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:10.090014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d56d5d19-5b1c-426a-ad45-ccb952a5c7c1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:10.091310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d56d5d19-5b1c-426a-ad45-ccb952a5c7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:10.093694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:10.199800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 10:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:10.354043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceb59b-5616-48b0-b04a-27406bb72f6a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:10.355539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-11ceb59b-5616-48b0-b04a-27406bb72f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:10.525766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ceb59b-5616-48b0-b04a-27406bb72f6a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:10.576831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:10.674929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:10.675539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:10.675822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:10.676145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:10.677837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:10.785461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:10.788654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e27fbb-a23a-4e25-aa2a-28e6757a515b service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:10.811735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 720/1440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:12.789999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61859-ff6d-4247-adcd-1e0c3eb2e33f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:12.789999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61859-ff6d-4247-adcd-1e0c3eb2e33f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:12.789999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c61859-ff6d-4247-adcd-1e0c3eb2e33f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:12.791655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:12.797955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cb69-d210-4b2a-88bd-883f7dafec51 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:12.798527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cb69-d210-4b2a-88bd-883f7dafec51 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:12.798696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7cb69-d210-4b2a-88bd-883f7dafec51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:12.845818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:12.884405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-10b7cb69-d210-4b2a-88bd-883f7dafec51 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:12.887197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-10b7cb69-d210-4b2a-88bd-883f7dafec51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:13.027520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:13.316853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61859-ff6d-4247-adcd-1e0c3eb2e33f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:13.318442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-90c61859-ff6d-4247-adcd-1e0c3eb2e33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:13.489546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c61859-ff6d-4247-adcd-1e0c3eb2e33f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:13.530617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:13.583263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:13.583759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:13.584055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:13.584346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:13.586019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:13.838561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:13.839988 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46054148-e6ff-4878-b918-d100b50dc13a service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:13.860631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 722/1443] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:16.100307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1c66-842f-476b-9a3f-fcd681d6ad8f None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:16.100901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1c66-842f-476b-9a3f-fcd681d6ad8f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:16.101026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f1c66-842f-476b-9a3f-fcd681d6ad8f None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:16.103099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:16.109198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84a24d-db8c-44be-b6d6-850255cd9ddc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:16.109835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84a24d-db8c-44be-b6d6-850255cd9ddc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:16.109835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84a24d-db8c-44be-b6d6-850255cd9ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:16.188332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cc84a24d-db8c-44be-b6d6-850255cd9ddc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:16.189603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cc84a24d-db8c-44be-b6d6-850255cd9ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:16.207032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:16.315647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:59:16.615573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f1c66-842f-476b-9a3f-fcd681d6ad8f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:16.617969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-427f1c66-842f-476b-9a3f-fcd681d6ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:16.769381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f1c66-842f-476b-9a3f-fcd681d6ad8f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:16.796296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:16.891589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:16.892145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:16.892429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:16.893079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:16.894492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:17.035059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:17.036564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc45703-6962-406f-9b93-a25bd8d84a79 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:17.056979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 723/1446] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:19.430184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a44f1-4efe-44a6-8a8e-b60fb216f30e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:19.430818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a44f1-4efe-44a6-8a8e-b60fb216f30e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:19.430818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a44f1-4efe-44a6-8a8e-b60fb216f30e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:19.432492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:19.438906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4d82-4588-4090-995b-fa52f3c99516 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:19.439531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4d82-4588-4090-995b-fa52f3c99516 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:19.439531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad4d82-4588-4090-995b-fa52f3c99516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:19.501217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:19.526357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5cad4d82-4588-4090-995b-fa52f3c99516 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:19.527648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5cad4d82-4588-4090-995b-fa52f3c99516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:19.638344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:19.933392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a44f1-4efe-44a6-8a8e-b60fb216f30e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:19.934925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-119a44f1-4efe-44a6-8a8e-b60fb216f30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:20.054434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a44f1-4efe-44a6-8a8e-b60fb216f30e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:20.099047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:20.157187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:20.158182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:20.158661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:20.159135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:20.162307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:20.330753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:20.332142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f702c0f1-815d-49e4-a189-6af66006ca5a service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:20.356225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 725/1449] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:22.484908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820d89f-3df8-4355-a936-5fbc6e0deb2b None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:22.485882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820d89f-3df8-4355-a936-5fbc6e0deb2b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:22.485882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820d89f-3df8-4355-a936-5fbc6e0deb2b None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:22.486975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:59:22.493189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf3a91-b364-473e-91f8-11f5bb37ca02 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:22.493732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf3a91-b364-473e-91f8-11f5bb37ca02 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:22.493893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bf3a91-b364-473e-91f8-11f5bb37ca02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:22.568755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b6bf3a91-b364-473e-91f8-11f5bb37ca02 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:22.570383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b6bf3a91-b364-473e-91f8-11f5bb37ca02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:22.593546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:22.678345 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:59:22.966460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820d89f-3df8-4355-a936-5fbc6e0deb2b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:22.968074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e820d89f-3df8-4355-a936-5fbc6e0deb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:23.095814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820d89f-3df8-4355-a936-5fbc6e0deb2b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:23.119163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:23.183031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:23.183768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:23.184142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:23.184460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:23.186228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:23.309878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:23.311358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf302c-2f97-4ff4-81f1-68c477719811 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:23.333359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 726/1452] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:25.613122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486dc1-4d16-4399-a8b8-74468bdbd5f5 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:25.613934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486dc1-4d16-4399-a8b8-74468bdbd5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:25.613934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486dc1-4d16-4399-a8b8-74468bdbd5f5 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:25.615554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:25.621953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15676df-664f-43a8-a566-b56e42ca6d4e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:25.622658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15676df-664f-43a8-a566-b56e42ca6d4e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:25.622658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15676df-664f-43a8-a566-b56e42ca6d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:25.692871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:25.697094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e15676df-664f-43a8-a566-b56e42ca6d4e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:25.699273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e15676df-664f-43a8-a566-b56e42ca6d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:25.870103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:26.160280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f486dc1-4d16-4399-a8b8-74468bdbd5f5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:26.162001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2f486dc1-4d16-4399-a8b8-74468bdbd5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:26.304861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f486dc1-4d16-4399-a8b8-74468bdbd5f5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:26.359205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:26.389921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:26.390534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:26.390811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:26.391082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:26.392760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:26.522931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:26.524341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828d07c-fb13-4f7b-b494-1b27acee844f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:26.545416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 728/1455] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:28.663667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76ca4-19c9-4570-b902-d42bcf14cd77 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:28.664335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76ca4-19c9-4570-b902-d42bcf14cd77 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:28.664335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e76ca4-19c9-4570-b902-d42bcf14cd77 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:28.665650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 10:59:28.673799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af01f8-b2d5-4719-8892-75f5e7975f57 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:28.674362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af01f8-b2d5-4719-8892-75f5e7975f57 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:28.674362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5af01f8-b2d5-4719-8892-75f5e7975f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:28.751742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d5af01f8-b2d5-4719-8892-75f5e7975f57 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:28.753025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d5af01f8-b2d5-4719-8892-75f5e7975f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:28.785489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:28.870573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 10:59:29.176382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76ca4-19c9-4570-b902-d42bcf14cd77 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:29.178701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-36e76ca4-19c9-4570-b902-d42bcf14cd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:29.301543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e76ca4-19c9-4570-b902-d42bcf14cd77 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:29.334188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:29.390915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:29.391944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:29.392464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:29.392955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:29.396064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:29.540752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:29.542229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7045a3d5-a25f-4516-84ad-ab840b05d477 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:29.563778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 729/1458] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:31.712538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2da27d-8038-4ca0-91ea-6737db3d965e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:31.713167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2da27d-8038-4ca0-91ea-6737db3d965e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:31.713477 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2da27d-8038-4ca0-91ea-6737db3d965e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:31.715329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:31.721420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31089-eea2-4eaa-9265-a715b23d456f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:31.721948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31089-eea2-4eaa-9265-a715b23d456f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:31.721948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e31089-eea2-4eaa-9265-a715b23d456f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:31.799290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-34e31089-eea2-4eaa-9265-a715b23d456f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:31.800558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-34e31089-eea2-4eaa-9265-a715b23d456f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:31.810216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:31.916858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:32.206856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2da27d-8038-4ca0-91ea-6737db3d965e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:32.208516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bc2da27d-8038-4ca0-91ea-6737db3d965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:32.380343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2da27d-8038-4ca0-91ea-6737db3d965e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:32.479414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:32.511204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:32.511823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:32.512123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:32.512392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:32.514028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:32.667740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:32.670527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1b1f2-7da9-4eac-806c-054b03ad2651 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:32.701554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 731/1461] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:34.877710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5da0f-16cc-47a1-9164-c9294eca2e87 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:34.878364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5da0f-16cc-47a1-9164-c9294eca2e87 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:34.878364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5da0f-16cc-47a1-9164-c9294eca2e87 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:34.879855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:34.886347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccecf4-98e5-4147-8989-4dbae1a51e0b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:34.886928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccecf4-98e5-4147-8989-4dbae1a51e0b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:34.886928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ccecf4-98e5-4147-8989-4dbae1a51e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:34.965515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a1ccecf4-98e5-4147-8989-4dbae1a51e0b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:34.966821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1ccecf4-98e5-4147-8989-4dbae1a51e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:35.014174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:35.092478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:35.387933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5da0f-16cc-47a1-9164-c9294eca2e87 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:35.389616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cbf5da0f-16cc-47a1-9164-c9294eca2e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:35.537468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5da0f-16cc-47a1-9164-c9294eca2e87 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:35.579179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:35.621247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:35.621977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:35.622302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:35.622620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:35.624371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:35.764726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:35.767364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d76cc-c605-4a3b-b19c-3b48d1e4c48e service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:35.804177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 732/1464] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:37.798102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce162a3-1729-4b22-9283-0788533adb7a None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.798672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce162a3-1729-4b22-9283-0788533adb7a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.798672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce162a3-1729-4b22-9283-0788533adb7a None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.800000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:37.807023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef8c58-f770-409c-93ec-ab5157480515 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:37.807585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef8c58-f770-409c-93ec-ab5157480515 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:37.808109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef8c58-f770-409c-93ec-ab5157480515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:37.930936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ceef8c58-f770-409c-93ec-ab5157480515 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:37.933089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ceef8c58-f770-409c-93ec-ab5157480515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:37.967153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:38.079383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:38.367651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce162a3-1729-4b22-9283-0788533adb7a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:38.369247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ce162a3-1729-4b22-9283-0788533adb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.486958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce162a3-1729-4b22-9283-0788533adb7a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:38.567732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:38.576977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:38.577590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:38.577867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:38.578171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:38.579824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:38.746109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:38.747570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25f6b3a-c0f9-46ce-aa23-c37b65bcb5bc service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:38.768023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 734/1467] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 10:59:40.788550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c513e-1fc1-4b41-807a-86ce0b015db4 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:40.788947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c513e-1fc1-4b41-807a-86ce0b015db4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:40.789224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c513e-1fc1-4b41-807a-86ce0b015db4 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:40.790967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 10:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 10:59:40.799068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ef55-9673-478c-8829-2fb47b96f897 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:40.799734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ef55-9673-478c-8829-2fb47b96f897 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:40.799734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242ef55-9673-478c-8829-2fb47b96f897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:40.882734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6242ef55-9673-478c-8829-2fb47b96f897 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 10:59:40.883973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6242ef55-9673-478c-8829-2fb47b96f897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:40.947056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:41.030829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 10:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 10:59:41.318019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c513e-1fc1-4b41-807a-86ce0b015db4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 10:59:41.321156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-966c513e-1fc1-4b41-807a-86ce0b015db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:41.463566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c513e-1fc1-4b41-807a-86ce0b015db4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 10:59:41.517656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:41.565827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 10:59:41.566787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 10:59:41.567269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 10:59:41.568361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 10:59:41.572208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 10:59:41.732748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 10:59:41.734215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45221da0-9f5d-4abb-8352-3859f8994858 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 10:59:41.754869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 735/1470] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 10:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 10:59:51.028779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=22 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 10:59:51.752478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=66 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:02.543962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65efc-fa32-418b-864a-08c18d0a65b0 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:02.544568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65efc-fa32-418b-864a-08c18d0a65b0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:02.544568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c65efc-fa32-418b-864a-08c18d0a65b0 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:02.545904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 736/1471] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:02.551511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1c40-13e8-4146-b64d-8f0a6f2a7333 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:02.552038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1c40-13e8-4146-b64d-8f0a6f2a7333 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:02.552300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf1c40-13e8-4146-b64d-8f0a6f2a7333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:02.623374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-28cf1c40-13e8-4146-b64d-8f0a6f2a7333 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:02.624637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-28cf1c40-13e8-4146-b64d-8f0a6f2a7333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:02.726835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 736/1472] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:03.027044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5710-a180-420c-ab1e-3a4312f90388 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:03.027359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5710-a180-420c-ab1e-3a4312f90388 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:03.027628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5710-a180-420c-ab1e-3a4312f90388 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:03.029087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 737/1473] 10.222.0.20 () {54 vars in 967 bytes} [Wed Aug 31 11:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:03.034775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6ca0b-7159-46e4-82cc-fbb06d831511 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:03.035051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6ca0b-7159-46e4-82cc-fbb06d831511 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:03.035563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6ca0b-7159-46e4-82cc-fbb06d831511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:03.151026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-77d6ca0b-7159-46e4-82cc-fbb06d831511 None None] MFA Rules not processed for user `72251fea0df54206b554236c14296a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:03.152335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-77d6ca0b-7159-46e4-82cc-fbb06d831511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:03.178247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:03.255693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 737/1474] 10.222.0.20 () {58 vars in 1064 bytes} [Wed Aug 31 11:00:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:03.263491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f5710-a180-420c-ab1e-3a4312f90388 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:03.265107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cf1f5710-a180-420c-ab1e-3a4312f90388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:03.407373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f5710-a180-420c-ab1e-3a4312f90388 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:03.534041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:03.534753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:03.535677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:03.536288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:03.538109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:03.601845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:03.668408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:03.669866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c29873-b228-408b-9b46-f300af1eafcf service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:03.690718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 738/1475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:00:05.862172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7dadb-7831-467e-877f-8c73998bd257 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:05.862900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7dadb-7831-467e-877f-8c73998bd257 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:05.862900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7dadb-7831-467e-877f-8c73998bd257 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:05.865270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 738/1476] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:05.872060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80e83a-fdc8-4c80-b53c-eee4d4ef584e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:05.872593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80e83a-fdc8-4c80-b53c-eee4d4ef584e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:05.873045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed80e83a-fdc8-4c80-b53c-eee4d4ef584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:05.990282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ed80e83a-fdc8-4c80-b53c-eee4d4ef584e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:05.992084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed80e83a-fdc8-4c80-b53c-eee4d4ef584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:06.149541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 739/1477] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:06.433336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7dadb-7831-467e-877f-8c73998bd257 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:06.434880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e7f7dadb-7831-467e-877f-8c73998bd257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:06.593144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7dadb-7831-467e-877f-8c73998bd257 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:06.650216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:06.707247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:06.707853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:06.708202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:06.708597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:06.709571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:06.711553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:00:06.711553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:00:06.711553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:00:06.714051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6b706-21a0-4e6f-a3fd-392fa9c81ad0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:06.735847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 739/1478] 10.222.0.20 () {56 vars in 1108 bytes} [Wed Aug 31 11:00:06 2022] GET /identity/v3/projects => generated 2553 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:00:08.860945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ac49-ce4e-450e-9992-1b8c9ed19997 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:08.861531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ac49-ce4e-450e-9992-1b8c9ed19997 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:08.861531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48ac49-ce4e-450e-9992-1b8c9ed19997 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:08.862878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 740/1479] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:08.869318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b139f53-709a-4d4a-b7a2-bb6874806c91 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:08.869919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b139f53-709a-4d4a-b7a2-bb6874806c91 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:08.869919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b139f53-709a-4d4a-b7a2-bb6874806c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:08.947979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0b139f53-709a-4d4a-b7a2-bb6874806c91 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:08.949267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0b139f53-709a-4d4a-b7a2-bb6874806c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:09.054088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 740/1480] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:09.352867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ac49-ce4e-450e-9992-1b8c9ed19997 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:09.354450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2a48ac49-ce4e-450e-9992-1b8c9ed19997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:09.365466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:09.516602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a48ac49-ce4e-450e-9992-1b8c9ed19997 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:09.615618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:09.651139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:09.651743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:09.652042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:09.652320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:09.653971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:09.757636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:09.759249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05137f-4153-4806-802e-f1c1f96f73bd service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:09.781599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 741/1481] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:00:11.983815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97437-9ed7-445c-8190-031a45582a90 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:11.984692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97437-9ed7-445c-8190-031a45582a90 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:11.984692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab97437-9ed7-445c-8190-031a45582a90 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:11.986688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 741/1482] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:00:11.992947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008dc05-4ac2-4d31-af8e-d7c149455c35 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:11.993864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008dc05-4ac2-4d31-af8e-d7c149455c35 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:11.993864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a008dc05-4ac2-4d31-af8e-d7c149455c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:12.075039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a008dc05-4ac2-4d31-af8e-d7c149455c35 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:12.076241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a008dc05-4ac2-4d31-af8e-d7c149455c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:12.134490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:12.200309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 742/1483] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:00:12.514827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97437-9ed7-445c-8190-031a45582a90 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:12.516701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4ab97437-9ed7-445c-8190-031a45582a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:12.542969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:12.663343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab97437-9ed7-445c-8190-031a45582a90 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:12.798015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:12.799056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:12.799547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:12.800070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:12.803745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:12.948564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:13.013756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:13.016480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006441a8-1f15-4ad1-ba08-ab106bc2fefe service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:13.214012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 742/1484] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:00:15.358621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ec7bd-3794-47c2-99f9-44360ef99622 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:15.359181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ec7bd-3794-47c2-99f9-44360ef99622 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:15.359181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ec7bd-3794-47c2-99f9-44360ef99622 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:15.360753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 743/1485] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:15.367877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf977-eba1-4ed7-96c6-de942fc165ed None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:15.368515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf977-eba1-4ed7-96c6-de942fc165ed None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:15.368622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bf977-eba1-4ed7-96c6-de942fc165ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:15.444554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-670bf977-eba1-4ed7-96c6-de942fc165ed None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:15.445783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-670bf977-eba1-4ed7-96c6-de942fc165ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:15.556261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 743/1486] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:15.851626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ec7bd-3794-47c2-99f9-44360ef99622 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:15.853236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-394ec7bd-3794-47c2-99f9-44360ef99622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:15.980325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ec7bd-3794-47c2-99f9-44360ef99622 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:16.058450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:16.061257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:16.061884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:16.062391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:16.062762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:16.064915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:16.190479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:16.193387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7577644-9628-45f6-90cf-5f4f56949da7 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:16.217596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 744/1487] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:00:18.519766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f26f-11df-45e0-b5cd-e1bb95a92128 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:18.520511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f26f-11df-45e0-b5cd-e1bb95a92128 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:18.520511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3f26f-11df-45e0-b5cd-e1bb95a92128 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:18.521825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 744/1488] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:18.528455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debbe03-81f0-4dc5-b2dc-5fca02fd8107 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:18.528948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debbe03-81f0-4dc5-b2dc-5fca02fd8107 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:18.529066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1debbe03-81f0-4dc5-b2dc-5fca02fd8107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:18.605401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1debbe03-81f0-4dc5-b2dc-5fca02fd8107 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:18.606650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1debbe03-81f0-4dc5-b2dc-5fca02fd8107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:18.663515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:18.712555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 745/1489] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:19.022615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f26f-11df-45e0-b5cd-e1bb95a92128 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:19.024194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a5a3f26f-11df-45e0-b5cd-e1bb95a92128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:19.063882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:19.144675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3f26f-11df-45e0-b5cd-e1bb95a92128 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:19.226598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:19.227208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:19.227490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:19.227762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:19.229440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:19.326131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:19.363819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:19.365305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d030e77-d306-4966-bc8b-a9ba5ba2e01a service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:19.384061 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 745/1490] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:00:21.717953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6f12-2d67-425c-9a4f-230b1666e73e None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:21.718564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6f12-2d67-425c-9a4f-230b1666e73e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:21.718564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f6f12-2d67-425c-9a4f-230b1666e73e None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:21.719859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 746/1491] 10.222.0.20 () {54 vars in 927 bytes} [Wed Aug 31 11:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:00:21.726959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7a6d-917a-4b35-9076-754eb58eb59e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:21.727528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7a6d-917a-4b35-9076-754eb58eb59e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:21.727710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b7a6d-917a-4b35-9076-754eb58eb59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:21.804317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-842b7a6d-917a-4b35-9076-754eb58eb59e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:00:21.805556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-842b7a6d-917a-4b35-9076-754eb58eb59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:21.915624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 746/1492] 10.222.0.20 () {58 vars in 1024 bytes} [Wed Aug 31 11:00:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:00:22.224738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f6f12-2d67-425c-9a4f-230b1666e73e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:00:22.226405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e9f6f12-2d67-425c-9a4f-230b1666e73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:22.358229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f6f12-2d67-425c-9a4f-230b1666e73e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:00:22.439811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:00:22.442417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:00:22.442719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:00:22.442987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:00:22.444762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:00:22.455128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:22.571891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:00:22.573334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbf2cc7-1fda-4b49-925a-242637e67e6e service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:00:22.593789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 747/1493] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:00:31.913777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=82 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:00:32.591120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=50 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:38.288544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:38.289156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:38.289156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:38.369074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:38.370309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:38.475619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 747/1494] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:38.484704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:38.485312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:38.485312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:38.569952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:38.571077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:38.668785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 748/1495] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:38.674013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:38.675215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:38.686392 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4c526-22dd-4afa-85c6-c2a72f314d73 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:38.747272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:38.782123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:38.782735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:38.783018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:38.783289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:38.783944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:01:38.784277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:38.785302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:38.785302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:01:38.785302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:01:38.786242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:38.798339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 748/1496] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:01:38.806670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:38.816147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb627c-87b3-4db0-b2e5-05b9aeb0bcf1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:38.877647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:38.912789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:38.913388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:38.913666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:38.913938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:38.914568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:38.916366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:38.916366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:01:38.916366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:01:38.917559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:38.973147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 749/1497] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:01:38 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:01:38.977480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:38.978962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.012474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e89204-180c-491e-85cd-fd2891406beb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:39.108710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:39.109370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.109646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.109913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.110552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:39.111562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:39.111562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:01:39.111562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:01:39.112775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:39.157979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 749/1498] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:01:38 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:01:39.162244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:39.163776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.226397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a4aaa9-e89f-435c-aabb-0bf55b96efb9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:39.312037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:39.323212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:39.323813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.324119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.324395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.325032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:39.326046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:39.326046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:01:39.326046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:01:39.326963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:39.339681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 750/1499] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:01:39.344403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:39.345953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.357397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca453eb-4b74-49ea-9059-174a96e18b56 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:39.383144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:39.454797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:39.455311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.455600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.455866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.456541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:39.457554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:39.457554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:01:39.457554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:01:39.458438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46592894-c372-4d3a-b48a-6b13e4a2d8c0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:39.472212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 750/1500] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:01:39.476636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:39.478219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.512614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc581a7-6ab8-4812-8671-de969f1155e6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:39.606702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:39.607383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.607659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.607930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] PATH_INFO: `/v3/projects/ab933e9f22f34bb6926d4c808f436fa5/users/b52223d91dee416bb21f67f1b676ae28/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.629667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab933e9f22f34bb6926d4c808f436fa5, user_id=b52223d91dee416bb21f67f1b676ae28, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:39.630704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:39.630704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:01:39.630704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:01:39.632692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd1860-8ccd-4830-bb67-bd61dd8236b6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:39.657660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 751/1501] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:01:39 2022] PUT /identity/v3/projects/ab933e9f22f34bb6926d4c808f436fa5/users/b52223d91dee416bb21f67f1b676ae28/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:01:39.669956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd730f0-ee13-47e8-bc08-8b60a5b79400 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.670352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd730f0-ee13-47e8-bc08-8b60a5b79400 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.670737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd730f0-ee13-47e8-bc08-8b60a5b79400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.729308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7dd730f0-ee13-47e8-bc08-8b60a5b79400 None None] MFA Rules not processed for user `b52223d91dee416bb21f67f1b676ae28`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:39.730450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7dd730f0-ee13-47e8-bc08-8b60a5b79400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.811516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 751/1502] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:01:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:39.824997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b02fb-15c2-495f-be82-b3e2da99f7b8 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.825279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b02fb-15c2-495f-be82-b3e2da99f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.825557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b02fb-15c2-495f-be82-b3e2da99f7b8 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.827443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 752/1503] 10.222.0.20 () {54 vars in 964 bytes} [Wed Aug 31 11:01:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:39.832465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4476d-f511-49c9-b88f-8983ff66461e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:39.832757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4476d-f511-49c9-b88f-8983ff66461e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:39.833037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4476d-f511-49c9-b88f-8983ff66461e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:39.864203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:39.910736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-14a4476d-f511-49c9-b88f-8983ff66461e None None] MFA Rules not processed for user `e1aaf56b3e654c0bb65d0c804fd797e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:39.911926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-14a4476d-f511-49c9-b88f-8983ff66461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:39.987142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 752/1504] 10.222.0.20 () {58 vars in 1061 bytes} [Wed Aug 31 11:01:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:01:39.996661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b02fb-15c2-495f-be82-b3e2da99f7b8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:39.998750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-154b02fb-15c2-495f-be82-b3e2da99f7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:40.049531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:40.105045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b02fb-15c2-495f-be82-b3e2da99f7b8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:40.177921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:40.178528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:40.178800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:40.179071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:40.180697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:40.278923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:40.280918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:40.284368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:40.299806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 753/1505] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:40.347777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfaa1e-2fe0-4f63-bde8-ab8f286850f6 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:40.348275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfaa1e-2fe0-4f63-bde8-ab8f286850f6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:40.348471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfaa1e-2fe0-4f63-bde8-ab8f286850f6 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:40.349654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 753/1506] 10.222.0.20 () {52 vars in 830 bytes} [Wed Aug 31 11:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:01:40.353453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:40.354670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:40.455656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377c309-9fe8-4ee6-bd98-329e9a52cef0 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:40.548742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:40.549423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:40.549537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:40.549802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] PATH_INFO: `/v3/users/b52223d91dee416bb21f67f1b676ae28` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:40.561794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b52223d91dee416bb21f67f1b676ae28)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:40.562817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:40.562817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:01:40.562817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:01:40.564305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:40.573769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:40.630083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b52223d91dee416bb21f67f1b676ae28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:40.630624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b52223d91dee416bb21f67f1b676ae28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:40.641970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b52223d91dee416bb21f67f1b676ae28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:40.665469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b52223d91dee416bb21f67f1b676ae28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:40.679312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b52223d91dee416bb21f67f1b676ae28'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:40.684829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 754/1507] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:01:40 2022] DELETE /identity/v3/users/b52223d91dee416bb21f67f1b676ae28 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:01:40.689982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:40.690293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:40.690567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:40.749028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:01:40.750372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:40.827041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:40.841485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 754/1508] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:01:40.853271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:40.854527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:40.956664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31a32-3785-4bee-8a5c-9da5adcaf741 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:41.011356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:41.033256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:41.033865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:41.034153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:41.034422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:41.036056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:41.151710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:41.153182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0b90d1-0285-4586-ba24-04fbb37eaa88 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:41.172846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 755/1509] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:01:41.512492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:01:41.514137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:01:41.529853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b44cad-e188-452b-973d-eb9abe869219 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:01:41.642434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:01:41.643014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:01:41.643329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:01:41.643608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] PATH_INFO: `/v3/projects/ab933e9f22f34bb6926d4c808f436fa5` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:01:41.655780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab933e9f22f34bb6926d4c808f436fa5)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:01:41.656892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:01:41.656892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:01:41.656892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:01:41.658255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:01:41.718567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab933e9f22f34bb6926d4c808f436fa5', 'request_id': 'req-d3296b43-2515-43ed-83c8-52a6c9fab23b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:41.731480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab933e9f22f34bb6926d4c808f436fa5', 'request_id': 'req-d3296b43-2515-43ed-83c8-52a6c9fab23b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:41.738843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab933e9f22f34bb6926d4c808f436fa5', 'request_id': 'req-d3296b43-2515-43ed-83c8-52a6c9fab23b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:41.766532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] The token cache is being invalidate because project ab933e9f22f34bb6926d4c808f436fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:01:41.766822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:41.767239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3296b43-2515-43ed-83c8-52a6c9fab23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:01:41.768696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 755/1510] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:01:41 2022] DELETE /identity/v3/projects/ab933e9f22f34bb6926d4c808f436fa5 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:01:51.170418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=62 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:01:51.764232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:12.175744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53351c60-35b2-49be-80d1-7650aecc4bed None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.176444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53351c60-35b2-49be-80d1-7650aecc4bed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.176444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53351c60-35b2-49be-80d1-7650aecc4bed None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.177695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:03:12.186222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0c93f-4aa6-438a-8441-76478a959e41 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.186849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0c93f-4aa6-438a-8441-76478a959e41 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.186849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0c93f-4aa6-438a-8441-76478a959e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.299278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ebd0c93f-4aa6-438a-8441-76478a959e41 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:12.300656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ebd0c93f-4aa6-438a-8441-76478a959e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.415127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:12.580913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53351c60-35b2-49be-80d1-7650aecc4bed None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:12.583092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-53351c60-35b2-49be-80d1-7650aecc4bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:12.730416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53351c60-35b2-49be-80d1-7650aecc4bed None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:12.736425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5d4e4-aad1-4916-9033-a32f0ea6c468 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:12.737078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d4e4-aad1-4916-9033-a32f0ea6c468 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:12.737364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d4e4-aad1-4916-9033-a32f0ea6c468 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:12.737643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5d4e4-aad1-4916-9033-a32f0ea6c468 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:12.747332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5d4e4-aad1-4916-9033-a32f0ea6c468 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:12.748976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5d4e4-aad1-4916-9033-a32f0ea6c468 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:12.760103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 757/1513] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:13.631721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895c7ef-e87e-4195-acd8-50f1dd36b925 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.632404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895c7ef-e87e-4195-acd8-50f1dd36b925 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.632513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895c7ef-e87e-4195-acd8-50f1dd36b925 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.634521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:03:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:13.640434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f3ed3-3f42-4a09-ba1f-e2c7a384d99f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.640948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f3ed3-3f42-4a09-ba1f-e2c7a384d99f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.640948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9f3ed3-3f42-4a09-ba1f-e2c7a384d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.661173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0f9f3ed3-3f42-4a09-ba1f-e2c7a384d99f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:13.662397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0f9f3ed3-3f42-4a09-ba1f-e2c7a384d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:13.673332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:13.831261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895c7ef-e87e-4195-acd8-50f1dd36b925 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:13.848625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895c7ef-e87e-4195-acd8-50f1dd36b925 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:13.855587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92887568-b2eb-4741-8096-792f10a39206 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:13.856297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92887568-b2eb-4741-8096-792f10a39206 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.856606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92887568-b2eb-4741-8096-792f10a39206 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.856941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92887568-b2eb-4741-8096-792f10a39206 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.867390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92887568-b2eb-4741-8096-792f10a39206 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:13.869611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92887568-b2eb-4741-8096-792f10a39206 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:13.879998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 758/1516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:13.912280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c57a3-1bdb-4abd-aebb-235130df8e82 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.912569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c57a3-1bdb-4abd-aebb-235130df8e82 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.912930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c57a3-1bdb-4abd-aebb-235130df8e82 None None] PATH_INFO: `/v3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:13.914133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 759/1517] 10.222.0.55 () {54 vars in 925 bytes} [Wed Aug 31 11:03:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:03:13.920641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46d08-068f-4456-90b4-ad1b348a713e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:13.920934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46d08-068f-4456-90b4-ad1b348a713e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:13.921208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c46d08-068f-4456-90b4-ad1b348a713e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:14.001335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e0c46d08-068f-4456-90b4-ad1b348a713e None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:14.002597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e0c46d08-068f-4456-90b4-ad1b348a713e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:14.022425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:14.087441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 759/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 31 11:03:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:17.871299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d502f5-4b7b-425d-86ca-fd1b135bad87 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.871926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d502f5-4b7b-425d-86ca-fd1b135bad87 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.871926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d502f5-4b7b-425d-86ca-fd1b135bad87 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.873304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 760/1519] 10.222.0.20 () {54 vars in 961 bytes} [Wed Aug 31 11:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:17.879205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd3445-0b29-4af8-9ece-c097e0452df1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.879886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd3445-0b29-4af8-9ece-c097e0452df1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.879886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd3445-0b29-4af8-9ece-c097e0452df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.905608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-44cd3445-0b29-4af8-9ece-c097e0452df1 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:17.906873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-44cd3445-0b29-4af8-9ece-c097e0452df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:17.917723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 760/1520] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Aug 31 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:17.925807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d502f5-4b7b-425d-86ca-fd1b135bad87 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:17.941473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d502f5-4b7b-425d-86ca-fd1b135bad87 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:17.947899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d5a28-93d1-47e7-9205-7890a32f8530 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:17.948653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d5a28-93d1-47e7-9205-7890a32f8530 service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:17.948939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d5a28-93d1-47e7-9205-7890a32f8530 service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:17.949217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d5a28-93d1-47e7-9205-7890a32f8530 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:17.958979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d5a28-93d1-47e7-9205-7890a32f8530 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:17.960666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d5a28-93d1-47e7-9205-7890a32f8530 service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:17.972969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 761/1521] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:25.122332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56657ba2-66eb-4b07-853b-c5e057eb80be None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:25.128450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56657ba2-66eb-4b07-853b-c5e057eb80be None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:25.128450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56657ba2-66eb-4b07-853b-c5e057eb80be None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:25.128450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:03:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:25.129563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155b8a3-85b3-4614-95fe-b71150486a5f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:25.129982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155b8a3-85b3-4614-95fe-b71150486a5f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:25.130107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155b8a3-85b3-4614-95fe-b71150486a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:25.149809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f155b8a3-85b3-4614-95fe-b71150486a5f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:25.150978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f155b8a3-85b3-4614-95fe-b71150486a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:25.158717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:25.313915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56657ba2-66eb-4b07-853b-c5e057eb80be None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:25.330264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56657ba2-66eb-4b07-853b-c5e057eb80be None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:25.335671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b68db-8c8d-470b-89ec-a298076d944d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:25.336292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b68db-8c8d-470b-89ec-a298076d944d service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:25.336573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b68db-8c8d-470b-89ec-a298076d944d service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:25.336844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b68db-8c8d-470b-89ec-a298076d944d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:25.345248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b68db-8c8d-470b-89ec-a298076d944d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:25.346817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b68db-8c8d-470b-89ec-a298076d944d service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:25.356952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 762/1524] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:26.193031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b0caf-32ca-4cdf-bac3-b17e9f5ded10 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:26.193877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b0caf-32ca-4cdf-bac3-b17e9f5ded10 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:26.193877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b0caf-32ca-4cdf-bac3-b17e9f5ded10 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:26.194916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 11:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:03:26.200546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462595fb-e636-446d-8cd4-fbb2fbb54ea4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:26.201036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462595fb-e636-446d-8cd4-fbb2fbb54ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:26.201149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462595fb-e636-446d-8cd4-fbb2fbb54ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:26.221831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-462595fb-e636-446d-8cd4-fbb2fbb54ea4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:26.223033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-462595fb-e636-446d-8cd4-fbb2fbb54ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:26.231178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 11:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:26.397361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b0caf-32ca-4cdf-bac3-b17e9f5ded10 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:26.413149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b0caf-32ca-4cdf-bac3-b17e9f5ded10 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:26.419159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e2418-2d7e-481e-b0ef-0e94344ff164 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:26.419763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2418-2d7e-481e-b0ef-0e94344ff164 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:26.420108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2418-2d7e-481e-b0ef-0e94344ff164 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:26.420386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e2418-2d7e-481e-b0ef-0e94344ff164 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:26.429988 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e2418-2d7e-481e-b0ef-0e94344ff164 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:26.431566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e2418-2d7e-481e-b0ef-0e94344ff164 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:26.441419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 764/1527] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:30.652900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a994e-a7f0-443b-9027-e44d3d2b4c24 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.653456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a994e-a7f0-443b-9027-e44d3d2b4c24 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.653456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a994e-a7f0-443b-9027-e44d3d2b4c24 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.654810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:30.660044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720cb31-1d25-46d8-9036-86397ad19f87 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:30.660509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720cb31-1d25-46d8-9036-86397ad19f87 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:30.660629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720cb31-1d25-46d8-9036-86397ad19f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:30.681357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6720cb31-1d25-46d8-9036-86397ad19f87 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:30.682822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6720cb31-1d25-46d8-9036-86397ad19f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:30.690822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:03:30.977977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a994e-a7f0-443b-9027-e44d3d2b4c24 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:30.980476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d23a994e-a7f0-443b-9027-e44d3d2b4c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:31.083532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a994e-a7f0-443b-9027-e44d3d2b4c24 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:31.088359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142dc654-ba9d-4b57-85cd-c679a652e7c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:31.089090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dc654-ba9d-4b57-85cd-c679a652e7c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:31.089520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dc654-ba9d-4b57-85cd-c679a652e7c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:31.089891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142dc654-ba9d-4b57-85cd-c679a652e7c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:31.099636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142dc654-ba9d-4b57-85cd-c679a652e7c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:31.101165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142dc654-ba9d-4b57-85cd-c679a652e7c9 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:31.111453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 765/1530] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:33.040466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aff29-4a1a-4831-b9c4-fa3a7bebc8ed None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.040996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aff29-4a1a-4831-b9c4-fa3a7bebc8ed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.040996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73aff29-4a1a-4831-b9c4-fa3a7bebc8ed None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.042332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:03:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:33.047784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5308f-31d1-4ce6-b8f7-581b90acbe29 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.048243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5308f-31d1-4ce6-b8f7-581b90acbe29 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.048380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5308f-31d1-4ce6-b8f7-581b90acbe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.068690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-29a5308f-31d1-4ce6-b8f7-581b90acbe29 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:33.069918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-29a5308f-31d1-4ce6-b8f7-581b90acbe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.077862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:33.371195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aff29-4a1a-4831-b9c4-fa3a7bebc8ed None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:33.372956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b73aff29-4a1a-4831-b9c4-fa3a7bebc8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:33.386338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73aff29-4a1a-4831-b9c4-fa3a7bebc8ed None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:33.390335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fae1f5-b691-4f6a-b6b5-74d17d4357f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:33.390975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fae1f5-b691-4f6a-b6b5-74d17d4357f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:33.391356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fae1f5-b691-4f6a-b6b5-74d17d4357f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:33.391776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fae1f5-b691-4f6a-b6b5-74d17d4357f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:33.402117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fae1f5-b691-4f6a-b6b5-74d17d4357f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:33.403544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fae1f5-b691-4f6a-b6b5-74d17d4357f6 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:33.413213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 767/1533] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:36.917247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29fd15-d406-4b15-9b9b-3dd3c01a41c5 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.917866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29fd15-d406-4b15-9b9b-3dd3c01a41c5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.917866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29fd15-d406-4b15-9b9b-3dd3c01a41c5 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.919176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:36.925291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8587e-0582-41f5-a2e3-dc0fdcd86cd1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:36.925843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8587e-0582-41f5-a2e3-dc0fdcd86cd1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:36.925843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8587e-0582-41f5-a2e3-dc0fdcd86cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:36.948209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-40c8587e-0582-41f5-a2e3-dc0fdcd86cd1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:36.949458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-40c8587e-0582-41f5-a2e3-dc0fdcd86cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:36.957456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:37.533349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29fd15-d406-4b15-9b9b-3dd3c01a41c5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:37.535020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3b29fd15-d406-4b15-9b9b-3dd3c01a41c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:37.599963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:37.614556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29fd15-d406-4b15-9b9b-3dd3c01a41c5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:37.619017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a18961-ff33-4f44-902e-52481e1e08de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:37.619625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a18961-ff33-4f44-902e-52481e1e08de service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:37.619902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a18961-ff33-4f44-902e-52481e1e08de service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:37.620212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a18961-ff33-4f44-902e-52481e1e08de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:37.629136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a18961-ff33-4f44-902e-52481e1e08de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:37.630554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a18961-ff33-4f44-902e-52481e1e08de service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:37.640373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 768/1536] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:39.642668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1c1f2-37fb-495b-8b09-51c98a413140 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.643242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1c1f2-37fb-495b-8b09-51c98a413140 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.643242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1c1f2-37fb-495b-8b09-51c98a413140 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.644520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 11:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:03:39.653173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a50c3-a0d7-4326-b493-c3d2d16ff35f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:39.653664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a50c3-a0d7-4326-b493-c3d2d16ff35f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:39.653798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a50c3-a0d7-4326-b493-c3d2d16ff35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:39.673454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6e7a50c3-a0d7-4326-b493-c3d2d16ff35f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:03:39.674637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6e7a50c3-a0d7-4326-b493-c3d2d16ff35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:39.682890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 11:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:03:40.238314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1c1f2-37fb-495b-8b09-51c98a413140 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:03:40.240145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6d1c1f2-37fb-495b-8b09-51c98a413140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:03:40.253343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1c1f2-37fb-495b-8b09-51c98a413140 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:03:40.256805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa06c5-7c6a-43be-989e-e6fb09e45c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:03:40.257395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa06c5-7c6a-43be-989e-e6fb09e45c79 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:03:40.257674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa06c5-7c6a-43be-989e-e6fb09e45c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:03:40.257945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa06c5-7c6a-43be-989e-e6fb09e45c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:03:40.267003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa06c5-7c6a-43be-989e-e6fb09e45c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:03:40.268456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa06c5-7c6a-43be-989e-e6fb09e45c79 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:03:40.278446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 770/1539] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:03:49.668417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=11 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:03:50.274240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:07.490175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.490815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.490917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.535491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:07.540073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.557826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 770/1540] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:07.568474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.571744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.571744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.623430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:07.625698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.650673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 771/1541] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:07.659045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.680832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819345ad-12e8-4630-8ecb-ba30555dece2 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:07.691618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:07.692632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.693076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.693672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.694902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:07.695452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:07.698822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca90170-9c5b-44b1-897c-68bd83ff2681 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:07.729184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 771/1542] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:07.748641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.767179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf8f70-1f94-47ff-8e5d-caae7236169f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.768228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf8f70-1f94-47ff-8e5d-caae7236169f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.768708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf8f70-1f94-47ff-8e5d-caae7236169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.778802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c75ac-1664-4411-92fb-775c441f6b39 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:07.807453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:07.808527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.809594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.809594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.810605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:07.823180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:07.875947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7ccf8f70-1f94-47ff-8e5d-caae7236169f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:07.878106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7ccf8f70-1f94-47ff-8e5d-caae7236169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:07.918949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 772/1543] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:07.930045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:07.931611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:07.933138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:07.966620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 772/1544] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/projects => generated 381 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:07.973690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:07.989377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:07.993131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.230093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f6046-80d1-417d-8515-eea64a616f47 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.258667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 773/1545] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:08.262957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.265563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.272173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.272750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.273270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.274279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.276137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:05:08.276137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:05:08.276137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:05:08.278698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377f71a-b17f-4814-9cc0-70fe3539213f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.301227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b48c2-fd6e-492a-809d-4bfeed3ea2e8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.319059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.319902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.320219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.320500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.321176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:08.321486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.323123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1514d5-4a9a-4b61-a838-bc254e235968 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.335224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 773/1546] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:07 2022] POST /identity/v3/users => generated 291 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:08.340277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.340614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.340826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.364508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 774/1547] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:08.366037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.366328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.366607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.409019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.410315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.419304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.420563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.429787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 774/1548] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.430651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.447900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 775/1549] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.449253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.461062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b183bea-919f-4f33-8008-669cd93f5953 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.479393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4010a-e5ce-426d-9512-5410c5c7afa1 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.482461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1395cfd-240e-48fe-bc9b-80fd49383bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.483286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1395cfd-240e-48fe-bc9b-80fd49383bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.483616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1395cfd-240e-48fe-bc9b-80fd49383bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.484111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1395cfd-240e-48fe-bc9b-80fd49383bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.485167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1395cfd-240e-48fe-bc9b-80fd49383bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.488889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1395cfd-240e-48fe-bc9b-80fd49383bf7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.500035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.500956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.501382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.502429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.503585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.504998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:05:08.504998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:05:08.504998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:05:08.506330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41c264-c802-4022-b84a-df0a7d54d4aa admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.524117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 775/1550] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:08.526351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.526832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.527293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.585827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 776/1551] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:08.587211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c7a3-6008-45e0-bda6-9d800803c814 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.587498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c7a3-6008-45e0-bda6-9d800803c814 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.587772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee8c7a3-6008-45e0-bda6-9d800803c814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.591749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.594253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.619520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-cee8c7a3-6008-45e0-bda6-9d800803c814 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.621081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cee8c7a3-6008-45e0-bda6-9d800803c814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.689644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 776/1552] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.690537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.697972 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 777/1553] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.699630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.700141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.700610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.705948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ecfad6-4b90-42b7-b146-9ec2beb0c8b7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.710906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98741e61-86bf-4fde-929d-45e72b5c9db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.711561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741e61-86bf-4fde-929d-45e72b5c9db5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.711841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741e61-86bf-4fde-929d-45e72b5c9db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.712286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741e61-86bf-4fde-929d-45e72b5c9db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.713165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741e61-86bf-4fde-929d-45e72b5c9db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.714567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741e61-86bf-4fde-929d-45e72b5c9db5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.733421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 777/1554] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:08.733421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41babb-cc52-4102-b018-ccec1b9441c6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.733421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41babb-cc52-4102-b018-ccec1b9441c6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.733421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e41babb-cc52-4102-b018-ccec1b9441c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.735321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.736923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.753161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 778/1555] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.760969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7e41babb-cc52-4102-b018-ccec1b9441c6 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.761223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.761939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7e41babb-cc52-4102-b018-ccec1b9441c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.776289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 778/1556] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.776289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.776289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.776289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.777024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e76025-afc8-4d87-b6f1-39f66ef73c75 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.791227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.791975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.792315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.792583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.793260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.795413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.813744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.816091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.824701 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 779/1557] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.825539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.826102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 779/1558] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/users => generated 290 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:08.827160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.847988 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf1841-3f9f-4f40-925b-9417c10faf9b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.849638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf174965-ab92-4c1e-9366-b411dba327e5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.855551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.856577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.857051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.857518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.858636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:08.859150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.859925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.860955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:05:08.860955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:05:08.860955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:05:08.862008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.862707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.863254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.864046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.865428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:08.866204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.868996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d045c5-39bf-42ca-86e8-f5dec8f7f874 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.873927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:08.875399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:08.894149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 780/1559] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:08.895129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.898454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 780/1560] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:08.900406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1756fd-6d2d-4390-ba90-ac1d3059df93 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.900933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1756fd-6d2d-4390-ba90-ac1d3059df93 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.901634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1756fd-6d2d-4390-ba90-ac1d3059df93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.906957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711314b-f70a-4ed4-8806-3d71215fe578 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:08.912763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:08.913603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.913888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.914164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] PATH_INFO: `/v3/projects/173d9af664764ba5bd694dab38d8bb6a/users/d0e5dae087184c56b1cb72fb7705d4d5/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.926651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fa1756fd-6d2d-4390-ba90-ac1d3059df93 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.928267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fa1756fd-6d2d-4390-ba90-ac1d3059df93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:08.932323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] RBAC: Authorizing `identity:create_grant(project_id=173d9af664764ba5bd694dab38d8bb6a, user_id=d0e5dae087184c56b1cb72fb7705d4d5, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:08.935309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:08.937643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 781/1561] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:08.942296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8064321-245e-414c-a003-6b80b31ad08c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:08.942296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8064321-245e-414c-a003-6b80b31ad08c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:08.942296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8064321-245e-414c-a003-6b80b31ad08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:08.959780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 781/1562] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:08 2022] PUT /identity/v3/projects/173d9af664764ba5bd694dab38d8bb6a/users/d0e5dae087184c56b1cb72fb7705d4d5/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:08.960432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:08.962943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d8064321-245e-414c-a003-6b80b31ad08c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:08.964415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d8064321-245e-414c-a003-6b80b31ad08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:09.040499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 782/1563] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:09.043569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.043569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.043569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.050306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35ec86-923e-4dcf-a437-ece31ce5854e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.056435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.057072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.057349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.057618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.058277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.059930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.068821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:09.070075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:09.072854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 782/1564] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:09.073598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.084396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 783/1565] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:09.085261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.094940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95df4-07bf-44b9-981a-abee964de191 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.095175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1c7d8-e97a-44a9-b0b0-070fbb060def None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.102330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b33c71-b59f-4a90-b02b-00c92aaf6c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.102725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.103289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b33c71-b59f-4a90-b02b-00c92aaf6c72 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.103680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.103787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b33c71-b59f-4a90-b02b-00c92aaf6c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.104180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b33c71-b59f-4a90-b02b-00c92aaf6c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.104304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.104624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.105269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b33c71-b59f-4a90-b02b-00c92aaf6c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.105772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.108787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b33c71-b59f-4a90-b02b-00c92aaf6c72 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.109487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.144859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 783/1566] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.145144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 784/1567] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.148665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.148861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.148861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.148861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.176696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:09.178206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:09.226370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f820a-e036-416c-a65f-9ecdb04bb66c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.233024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.233645 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.233918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.234194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.234846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:09.235220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.237097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9ddff-f9d3-41f7-991e-275b301aed6d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.249823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 784/1568] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:09.251110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.251404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.252502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.269465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 785/1569] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:09.270609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.271836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:09.273132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:09.281539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 785/1570] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:09.282399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.291526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746be664-ccf5-41d2-911d-92bab556d4e7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.292776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68480a1-ef25-4240-b70a-57223f05553e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.296671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.297300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.297632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.297764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.297865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.298319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.298600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:09.299288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.299288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.299478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.299573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.301200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.301420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.313065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 786/1571] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:09.314197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.329359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 786/1572] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:09.329359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.330173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7768f-04f6-489c-ae92-76e71df18c5a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.331275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.331903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.332224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.332504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.333496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:09.333910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.335924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.338850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd7a80-deb6-44bd-a9e8-64472c3e086f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.346355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.346970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.347249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.347517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.348194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.350012 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 787/1573] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:09.350198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.350805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.362900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abc1c3-7417-416d-aa6f-a72809256025 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.368375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ed66f-f9b2-4e92-9851-77c4dd1c3192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.368995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ed66f-f9b2-4e92-9851-77c4dd1c3192 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.369272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ed66f-f9b2-4e92-9851-77c4dd1c3192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.369544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ed66f-f9b2-4e92-9851-77c4dd1c3192 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.370180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ed66f-f9b2-4e92-9851-77c4dd1c3192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.372164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ed66f-f9b2-4e92-9851-77c4dd1c3192 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.374730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 787/1574] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.375554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.385977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:09.388591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a489d-d9e0-4ef7-bdf3-fbe201e8902d admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.395299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.396115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.396400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.396674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.397368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.399287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.401335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 788/1575] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/users => generated 284 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.403839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.404357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.404835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.427231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:09.428770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:09.433803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 788/1576] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.434822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.439317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 789/1577] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:09.440292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.450124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:09.450709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c8e9f-a82c-42a6-8276-f388886ffff9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.502607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673c585-ebc2-4d93-ad9f-61e4df92de37 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.506905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.507525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.507815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.508126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.513043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.513707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.513986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.514257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] PATH_INFO: `/v3/projects/0ee7c5a0b37b4192a3746ee760ecefa5/users/0517d50b405e4bce991bb22eef6076a4/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.531373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee7c5a0b37b4192a3746ee760ecefa5, user_id=0517d50b405e4bce991bb22eef6076a4, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.534126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.548976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 790/1578] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:09 2022] PUT /identity/v3/projects/0ee7c5a0b37b4192a3746ee760ecefa5/users/0517d50b405e4bce991bb22eef6076a4/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:09.549987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.572250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.573697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.583668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 789/1579] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:09.584459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.594878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58737e43-b871-489c-8ae9-af3f171f3940 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.618460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa420bc-15b8-4974-ab76-4451276314bc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.624347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.624973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.625307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.625650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.626357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.628758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.669437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 791/1580] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/projects => generated 371 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.670419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.676734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.677357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.677901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.678176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.678828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.680980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.715409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 790/1581] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.716760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.741482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:09.747889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3cfae7-06b4-4a29-8504-590b00398261 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.780190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:09.812524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ca99-09bc-42c9-90f4-4a7be1fa6fe3 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.821154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.822159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.822660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.823112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.824143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.826954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.848253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 791/1582] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:09.848253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.856960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.857621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.857971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.858350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.859059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.860972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.867346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32bca3-092a-4ab0-be6e-85a900df11c9 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.873839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75685e4-5e57-4b02-b2bb-a42c2f182bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.874590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75685e4-5e57-4b02-b2bb-a42c2f182bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.875142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75685e4-5e57-4b02-b2bb-a42c2f182bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.875291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75685e4-5e57-4b02-b2bb-a42c2f182bc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.875977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75685e4-5e57-4b02-b2bb-a42c2f182bc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.879857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 792/1583] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:09.879857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.880147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75685e4-5e57-4b02-b2bb-a42c2f182bc1 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.895213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fc68b3-29cf-4152-b1a6-08bf964fa36c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.903739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.904821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.905327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.905798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.910172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 792/1584] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/users => generated 294 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:09.910358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:09.910358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.910538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.910538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.910538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.911895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.933607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:09.935020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:09.937375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 793/1585] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:09.938906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.951692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 793/1586] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:09.957233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:09.961325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c2bb-cc14-48fb-a5f6-0aca9c5524b7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.968160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.968791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c33ad6-bae9-41c4-a9ca-123323112d8e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:09.969135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.969601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.970091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.972058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.973408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:09.976852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:09.977554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:09.977843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:09.978146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:09.978974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:09.981501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.009729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 794/1587] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/users => generated 286 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:10.010754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.019813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 794/1588] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:09 2022] POST /identity/v3/users => generated 303 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:10.021564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.022985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e114c53-0a45-4933-8ed2-63a6b78cc72f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.027876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.028615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.028980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.029341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.030141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.031739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.039149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef68cc9-a6b7-4c5a-8abb-f80f96c110de admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.048336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 795/1589] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.048336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.052882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.053579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.053861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.054195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.054933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.056718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.068692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d721ed-c0d7-4d26-b74c-59a044830d24 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.071292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 795/1590] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.072165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.077981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.080295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.081827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.082319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.083004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.086034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.088037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c2723-1083-4ce5-a526-02a58e7fcb3e admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.096270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.096844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.097182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.097542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.098246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.099368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 796/1591] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.100436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.101178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.148056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 796/1592] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:10 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:10.149263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.180084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2c5978-365f-4e0e-b15c-bbd70cb287ea admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.253380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.254379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.254837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.255341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.255534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce284db-94a0-4a5f-a161-48f9baa60e19 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.266163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.269429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.270238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.270687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.271186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.272881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.273376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.275452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.288874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 797/1593] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:10.290060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.296741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 797/1594] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.298097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.303048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:10.311479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71c302-c7ce-4da2-a97f-85b9c59f2590 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.316396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee082db2-2e53-4fd4-a510-d0800a7db629 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.325873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.326248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.326889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.327068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.327348 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.327712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.327876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] PATH_INFO: `/v3/projects/7218420157c142878e694a8379e0113f/users/ba116e33912e46abb0a1e732e05b14b7/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.328304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.330708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.335560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.359186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 798/1595] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.359186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.367409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7218420157c142878e694a8379e0113f, user_id=ba116e33912e46abb0a1e732e05b14b7, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.369423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:05:10.369423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:05:10.369423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:05:10.372183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a53389-ec5c-4573-a3d9-caa274a788d0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.373573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.381239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.382399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.382952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.383395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] PATH_INFO: `/v3/projects/9ceec51e430e4a00b407068a2243be63/users/19643be9952649aba81e61d5d55290c4/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.398487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 798/1596] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:10 2022] PUT /identity/v3/projects/7218420157c142878e694a8379e0113f/users/ba116e33912e46abb0a1e732e05b14b7/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:10.399737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.423438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ceec51e430e4a00b407068a2243be63, user_id=19643be9952649aba81e61d5d55290c4, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.427694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.453171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 799/1597] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:10 2022] PUT /identity/v3/projects/9ceec51e430e4a00b407068a2243be63/users/19643be9952649aba81e61d5d55290c4/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:10.455466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.491734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:10.526995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d79f76-eb0b-4892-a9ac-6a61fd4ce956 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.555416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d0b9e7-4bb4-4049-8642-73aa00e84542 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.562901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.564050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.564764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.565294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.566436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.570152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.610591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 800/1598] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:10 2022] POST /identity/v3/users => generated 284 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:10.612416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.629810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f631eb-77b2-44cc-b560-dd3d38ea6a4e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.641108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.641575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e6686-72d2-4dba-a8fd-2b0ee938338e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.643313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.643488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6686-72d2-4dba-a8fd-2b0ee938338e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.643891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.644018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6686-72d2-4dba-a8fd-2b0ee938338e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.644166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.644553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e6686-72d2-4dba-a8fd-2b0ee938338e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.645267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.646045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e6686-72d2-4dba-a8fd-2b0ee938338e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.648238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.649675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e6686-72d2-4dba-a8fd-2b0ee938338e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.662310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 799/1599] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.663149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.668276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 801/1600] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.668276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.668276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.668276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.677230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0fe157-3607-4412-94d2-b1a6e434eeed admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.685831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32277c-f3ae-4602-8873-0737fce702d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.686499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32277c-f3ae-4602-8873-0737fce702d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.686785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32277c-f3ae-4602-8873-0737fce702d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.687121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32277c-f3ae-4602-8873-0737fce702d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.688152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32277c-f3ae-4602-8873-0737fce702d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.691343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f32277c-f3ae-4602-8873-0737fce702d5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.709241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] MFA Rules not processed for user `ba116e33912e46abb0a1e732e05b14b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:10.710804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:10.723816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 800/1601] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.725600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.726595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.726955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.760258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:10.762177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:10.772539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 801/1602] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:10.773278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.791439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09369ee0-8376-4497-9f09-e6db477a3eb9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.798080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.801416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.802844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.804134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.805650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.810776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.832279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 802/1603] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:10.833238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.845429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb33d2-ca98-49ee-b232-5bc5612e173e admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.847175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 802/1604] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:10.848195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.854318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.855513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.856031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.856642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] PATH_INFO: `/v3/projects/a25cd48f364541dbaceabdac555bb343/users/714872990c384663bcd564cd5b9ae432/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.864045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3758ed-fb32-45bb-90fc-5af6b5d783db None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:10.868551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:10.869733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:10.870015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:10.870457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] PATH_INFO: `/v3/projects/56f3b58753d74bf3bd4e394c2e4a6868/users/edc6faa3e7384671b30e9b2e3bbcb5ad/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:10.880117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25cd48f364541dbaceabdac555bb343, user_id=714872990c384663bcd564cd5b9ae432, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.883632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.897425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] RBAC: Authorizing `identity:create_grant(project_id=56f3b58753d74bf3bd4e394c2e4a6868, user_id=edc6faa3e7384671b30e9b2e3bbcb5ad, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:10.903246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:10.908558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 803/1605] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:10 2022] PUT /identity/v3/projects/a25cd48f364541dbaceabdac555bb343/users/714872990c384663bcd564cd5b9ae432/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:10.914093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.933471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 803/1606] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:10 2022] PUT /identity/v3/projects/56f3b58753d74bf3bd4e394c2e4a6868/users/edc6faa3e7384671b30e9b2e3bbcb5ad/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:10.934440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:10.964359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:11.038290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312aa1a6-0823-4314-b8b6-281838cc89de admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.046456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd784c1c-5ee6-4ace-9df0-b93449fd41dc admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.047331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:11.048485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.049644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.049644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.100712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:11.143056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:11.144374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.144891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.145459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] PATH_INFO: `/v3/projects/9f40536d651748db8bbb8d8630ab5221/users/80361bb4875b4bb8901a0ed5b945eb1f/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.169779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f40536d651748db8bbb8d8630ab5221, user_id=80361bb4875b4bb8901a0ed5b945eb1f, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:11.173160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:11.199452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 804/1607] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:10 2022] PUT /identity/v3/projects/9f40536d651748db8bbb8d8630ab5221/users/80361bb4875b4bb8901a0ed5b945eb1f/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:11.201197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:11.218261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:11.226212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:11.268102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 804/1608] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:11.270335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:11.287168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d1443-0d12-44f3-950e-14f217767132 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.296946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3c43d-0c06-47a7-bfd5-ee90796e43f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:11.299559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c43d-0c06-47a7-bfd5-ee90796e43f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.300330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c43d-0c06-47a7-bfd5-ee90796e43f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.301206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3c43d-0c06-47a7-bfd5-ee90796e43f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.305643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c43d-0c06-47a7-bfd5-ee90796e43f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:11.307625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3c43d-0c06-47a7-bfd5-ee90796e43f3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:11.334564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 805/1609] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:11.336091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07928145-fe8c-4abf-a510-2372e675e981 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.336382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07928145-fe8c-4abf-a510-2372e675e981 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.336660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07928145-fe8c-4abf-a510-2372e675e981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.394053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-07928145-fe8c-4abf-a510-2372e675e981 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:11.396507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-07928145-fe8c-4abf-a510-2372e675e981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:11.416338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2a280-8aea-4a8b-b95d-17b475a6e27a service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.419007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 806/1610] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:11.421018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19b979-0fff-42b9-81a6-509d2fb504ef None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.421018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19b979-0fff-42b9-81a6-509d2fb504ef None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.421269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19b979-0fff-42b9-81a6-509d2fb504ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.497050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1d19b979-0fff-42b9-81a6-509d2fb504ef None None] MFA Rules not processed for user `edc6faa3e7384671b30e9b2e3bbcb5ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:11.497050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1d19b979-0fff-42b9-81a6-509d2fb504ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:11.513857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:11.514624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.514998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.515371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.601410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:11.611951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 807/1611] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:11.613512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.613880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.614896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.649552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:11.652118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:11.660038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:11.661828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:11.669222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 808/1612] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:11.680164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:11.682426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 805/1613] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:11.698198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41676a42-f842-4ae1-b4a1-4e02c8786a53 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.698536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:11.703662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:11.704434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.705595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.705920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] PATH_INFO: `/v3/projects/26f5f9d4c25e4db3aa3d08b10a247f11/users/ff2561c82e5c4015ae0bd1cf2f09cf1d/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:11.729082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=26f5f9d4c25e4db3aa3d08b10a247f11, user_id=ff2561c82e5c4015ae0bd1cf2f09cf1d, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:11.731886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:11.760242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 809/1614] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:11 2022] PUT /identity/v3/projects/26f5f9d4c25e4db3aa3d08b10a247f11/users/ff2561c82e5c4015ae0bd1cf2f09cf1d/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:11.761461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:11.807357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:11.860448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d64606-d41d-467b-9bd6-37895680ae1c service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.983695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2f09e-41fb-47bb-aaf1-7060bcc9f4bc admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:11.994696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:11.995551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:11.996136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:11.996271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.007143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507515f-9f8e-46e6-aaa4-7fc12deab463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.008410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507515f-9f8e-46e6-aaa4-7fc12deab463 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.008410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507515f-9f8e-46e6-aaa4-7fc12deab463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.013234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507515f-9f8e-46e6-aaa4-7fc12deab463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.015588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507515f-9f8e-46e6-aaa4-7fc12deab463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.018132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507515f-9f8e-46e6-aaa4-7fc12deab463 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.018717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.020979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.056867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 806/1615] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:12.058286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.081293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ed56f-77f3-4c3d-8d29-19a3ff3fd86b service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.085622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 810/1616] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:11 2022] POST /identity/v3/projects => generated 371 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:12.088728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.089381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.089812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.157664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] MFA Rules not processed for user `ff2561c82e5c4015ae0bd1cf2f09cf1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:12.169216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:12.207031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.208135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.210254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.211964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.322754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:12.342677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 811/1617] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:12.345897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.352673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.357854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.375377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9384a9-0f52-4034-a579-cdbec6f9b130 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.389629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 807/1618] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:12.390628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.391213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.392942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.393326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.393720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.395331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.402606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.418600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea96b8-7a82-4602-a3e3-0ad06db538dd service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.436170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.437155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.440468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.440468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.444666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.446924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.482413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 812/1619] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:12 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:12.492543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.510090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 808/1620] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:12 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:12.514301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.559255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911aa1a-5d1d-44b8-83a0-9c6ce41923ea admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.681052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b9618-3c30-47b0-aa33-ff2a912e0a52 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.696952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.699006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.699476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.700462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.707552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.713419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.735142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.737408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.737901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.738433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.739318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.741561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.757808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:12.778673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 809/1621] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:12 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:12.787764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.805340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 813/1622] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:12.806569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.826735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795741-1af9-4a9c-a480-870e5ca1ed22 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.837837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03643cd2-c2b0-4158-aac7-04f825432718 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:12.843558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.844737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.845748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.846242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] PATH_INFO: `/v3/projects/f15821c9ec314eda9af3cc72fc281496/users/c3e7c2fd7501436abaf63c8978124fb4/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.853641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:12.855938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:12.859189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:12.859750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:12.863122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.892389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.925587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15821c9ec314eda9af3cc72fc281496, user_id=c3e7c2fd7501436abaf63c8978124fb4, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:12.931434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:12.993443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 810/1623] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:12.993443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.994984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 814/1624] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:12 2022] PUT /identity/v3/projects/f15821c9ec314eda9af3cc72fc281496/users/c3e7c2fd7501436abaf63c8978124fb4/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:12.994984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:12.999775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3c80-61bf-4ccc-af68-ab5d1495b5a0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:13.182074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:13.182722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.183003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.183577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] PATH_INFO: `/v3/projects/a3e50084410944d283a7a24709c93e72/users/bd8a1d38c8fb4f678cc63795820fc31c/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.208646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03d262-2629-4e4b-b19e-a30f02ff6d53 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:13.218437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e50084410944d283a7a24709c93e72, user_id=bd8a1d38c8fb4f678cc63795820fc31c, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:13.219410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:13.221889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.222742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:13.223045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.225598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.226475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:13.228491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:13.278822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 815/1625] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:12 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:13.279901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:13.289052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 811/1626] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:12 2022] PUT /identity/v3/projects/a3e50084410944d283a7a24709c93e72/users/bd8a1d38c8fb4f678cc63795820fc31c/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:13.297324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:13.316087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575bba-e21d-4bc0-8c75-35e0a8d46e08 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:13.359967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:13.463000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f076005-e52c-4c49-8571-daa88336a5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:13.463889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f076005-e52c-4c49-8571-daa88336a5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.464248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f076005-e52c-4c49-8571-daa88336a5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.464716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f076005-e52c-4c49-8571-daa88336a5a9 admin admin] PATH_INFO: `/v3/projects/f15821c9ec314eda9af3cc72fc281496/users/c3e7c2fd7501436abaf63c8978124fb4/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.467485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f076005-e52c-4c49-8571-daa88336a5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15821c9ec314eda9af3cc72fc281496, user_id=c3e7c2fd7501436abaf63c8978124fb4, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:13.474601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f076005-e52c-4c49-8571-daa88336a5a9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:13.491873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 816/1627] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:13 2022] PUT /identity/v3/projects/f15821c9ec314eda9af3cc72fc281496/users/c3e7c2fd7501436abaf63c8978124fb4/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:13.492635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60cbe4-0bc5-4e47-b3a2-db2af030c2ff admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:13.503078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ba18-c88d-4b0e-9a50-48522c199d2a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.503368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ba18-c88d-4b0e-9a50-48522c199d2a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.503772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69ba18-c88d-4b0e-9a50-48522c199d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.569977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6b69ba18-c88d-4b0e-9a50-48522c199d2a None None] MFA Rules not processed for user `c3e7c2fd7501436abaf63c8978124fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:13.571506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6b69ba18-c88d-4b0e-9a50-48522c199d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:13.624128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:13.624763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.625047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.625337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.626014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:13.631296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:13.653561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 812/1628] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:13.654623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:13.662603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:13.674208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee44f8-123f-412e-b69a-02409ef8ae8f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:13.692964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:13.693631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.693836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.694115 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] PATH_INFO: `/v3/projects/a3e50084410944d283a7a24709c93e72/users/bd8a1d38c8fb4f678cc63795820fc31c/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.707141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e50084410944d283a7a24709c93e72, user_id=bd8a1d38c8fb4f678cc63795820fc31c, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:13.713170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:13.735060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 817/1629] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:13.735675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 813/1630] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:13 2022] PUT /identity/v3/projects/a3e50084410944d283a7a24709c93e72/users/bd8a1d38c8fb4f678cc63795820fc31c/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:13.752871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.753676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.756264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:13.774667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:13.896270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] MFA Rules not processed for user `bd8a1d38c8fb4f678cc63795820fc31c`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:13.909592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:13.914896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebca069-50cf-442e-9a91-af5395db7245 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:13.945864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6d40b-52c7-4114-a004-d672edc916b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:13.947983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d40b-52c7-4114-a004-d672edc916b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:13.956952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d40b-52c7-4114-a004-d672edc916b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:13.957525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6d40b-52c7-4114-a004-d672edc916b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:14.187242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6d40b-52c7-4114-a004-d672edc916b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:14.189620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6d40b-52c7-4114-a004-d672edc916b8 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:14.190985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 818/1631] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:14.222166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 814/1632] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:05:14.257709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:14.288291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50a9a1-cf34-4be0-b4f5-5b5d32a0b2ed None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:14.295608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a1da0-f85a-472d-b41f-7cb90cc121bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:14.297475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a1da0-f85a-472d-b41f-7cb90cc121bf service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:14.298005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a1da0-f85a-472d-b41f-7cb90cc121bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:14.299583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a1da0-f85a-472d-b41f-7cb90cc121bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:14.326461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a1da0-f85a-472d-b41f-7cb90cc121bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:14.335001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a1da0-f85a-472d-b41f-7cb90cc121bf service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:14.389704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 819/1633] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:14.638810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb95d2-2706-4c1d-9e61-22805eed6f08 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:14.639633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb95d2-2706-4c1d-9e61-22805eed6f08 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:14.640160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb95d2-2706-4c1d-9e61-22805eed6f08 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:14.643750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 815/1634] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 11:05:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:14.661934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acdbda-9109-4e3a-836b-bdd704116688 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:14.662263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acdbda-9109-4e3a-836b-bdd704116688 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:14.662495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29acdbda-9109-4e3a-836b-bdd704116688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:14.737176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-29acdbda-9109-4e3a-836b-bdd704116688 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:14.742437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-29acdbda-9109-4e3a-836b-bdd704116688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:14.873038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 820/1635] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:14.874839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3e169-04f2-40b9-8207-2028ac9010ef None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:14.875852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3e169-04f2-40b9-8207-2028ac9010ef None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:14.876948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3e169-04f2-40b9-8207-2028ac9010ef None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:14.879096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 816/1636] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 11:05:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:14.887753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:14.888088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:14.888375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:14.915653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:14.917199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:14.942116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 821/1637] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:05:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:14.991884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3e169-04f2-40b9-8207-2028ac9010ef None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:15.007403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:15.025791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3e169-04f2-40b9-8207-2028ac9010ef None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:15.030388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8b863-6e22-4568-81e1-6145bb05a557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:15.031444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8b863-6e22-4568-81e1-6145bb05a557 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:15.032119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8b863-6e22-4568-81e1-6145bb05a557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:15.033309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8b863-6e22-4568-81e1-6145bb05a557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:15.068473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8b863-6e22-4568-81e1-6145bb05a557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:15.070457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8b863-6e22-4568-81e1-6145bb05a557 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:15.087112 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 817/1638] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:15.594556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:15.704551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:15.781488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a92a2-67df-4e8e-a54d-32429eada5d6 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:15.791626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:15.792494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:15.792799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:15.793077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] PATH_INFO: `/v3/users/edc6faa3e7384671b30e9b2e3bbcb5ad` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:15.798930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc6faa3e7384671b30e9b2e3bbcb5ad)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:15.809773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: WARNING py.warnings [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:05:15.809773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: warnings.warn(msg) Aug 31 11:05:15.809773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: Aug 31 11:05:15.812613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:15.962751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc6faa3e7384671b30e9b2e3bbcb5ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:15.985385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc6faa3e7384671b30e9b2e3bbcb5ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:16.009618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc6faa3e7384671b30e9b2e3bbcb5ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:16.082687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc6faa3e7384671b30e9b2e3bbcb5ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:16.083946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc6faa3e7384671b30e9b2e3bbcb5ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:16.104626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 822/1639] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:15 2022] DELETE /identity/v3/users/edc6faa3e7384671b30e9b2e3bbcb5ad => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:16.109617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e27362-ddfd-4a3e-b421-5647d3509608 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.111329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e27362-ddfd-4a3e-b421-5647d3509608 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.111808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e27362-ddfd-4a3e-b421-5647d3509608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:16.141295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-70e27362-ddfd-4a3e-b421-5647d3509608 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:16.143188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-70e27362-ddfd-4a3e-b421-5647d3509608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:16.229272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 818/1640] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:16.276463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:16.278409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:16.283361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.284099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.284530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:16.349810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] MFA Rules not processed for user `80361bb4875b4bb8901a0ed5b945eb1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:16.352995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:16.425701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6534fe6-de15-4727-adae-7b42014892ca admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:16.437997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:16.438770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.439481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.440701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:16.466560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:16.468494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:16.495745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 823/1641] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:16.506001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 819/1642] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:16 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:16.511637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:16.517725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:16.549424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5b8e8e-24ae-4161-b212-1a08f4ff1368 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:16.556489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:16.557102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.557708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.558109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:16.558864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:16.568946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:16.624243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 824/1643] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:16 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:16.625955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:16.642090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df9ea1-1484-4fff-9d78-335e8ea9a80a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:16.848110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:16.848759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.849030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.849304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:16.850165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:16.852298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:16.901230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 820/1644] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:16 2022] POST /identity/v3/users => generated 308 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:16.904951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:16.918393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e6496-42da-4f02-9add-c4744db51614 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:16.934122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:16.934927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.935206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.935492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:16.936315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:16.938064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:16.968194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 825/1645] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:16.972106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:16.991099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f80b19-dfb1-468d-bcf4-b4a07e31b54a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:16.998489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:16.999169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:16.999450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:16.999718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] PATH_INFO: `/v3/projects/5afe8fdbf8764e96a7dee56746c08c1e/users/00ff5fd2e6ee4621bf9c9158d2724071/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.009236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:17.037797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afe8fdbf8764e96a7dee56746c08c1e, user_id=00ff5fd2e6ee4621bf9c9158d2724071, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:17.040740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:17.077025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 821/1646] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:16 2022] PUT /identity/v3/projects/5afe8fdbf8764e96a7dee56746c08c1e/users/00ff5fd2e6ee4621bf9c9158d2724071/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:17.077994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:17.088908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770906f8-e03d-4d75-859e-b10cf2cde226 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:17.141165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:17.279321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256750ca-dc51-452e-96c5-a6c7bb87d53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:17.279960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256750ca-dc51-452e-96c5-a6c7bb87d53b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.280284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256750ca-dc51-452e-96c5-a6c7bb87d53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.280547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256750ca-dc51-452e-96c5-a6c7bb87d53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.281211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256750ca-dc51-452e-96c5-a6c7bb87d53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:17.283444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256750ca-dc51-452e-96c5-a6c7bb87d53b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:17.310570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:17.313177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 826/1647] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:17.328134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71a8ec1-3e9e-404f-ba47-7f7678a7c607 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:17.339354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da43ce-5ffc-45bd-8186-880a01a311e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:17.340070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da43ce-5ffc-45bd-8186-880a01a311e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.340358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da43ce-5ffc-45bd-8186-880a01a311e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.340637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da43ce-5ffc-45bd-8186-880a01a311e1 admin admin] PATH_INFO: `/v3/projects/5afe8fdbf8764e96a7dee56746c08c1e/users/00ff5fd2e6ee4621bf9c9158d2724071/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.346153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da43ce-5ffc-45bd-8186-880a01a311e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afe8fdbf8764e96a7dee56746c08c1e, user_id=00ff5fd2e6ee4621bf9c9158d2724071, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:17.364793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da43ce-5ffc-45bd-8186-880a01a311e1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:17.421566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 822/1648] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:17 2022] PUT /identity/v3/projects/5afe8fdbf8764e96a7dee56746c08c1e/users/00ff5fd2e6ee4621bf9c9158d2724071/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:17.659266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f764f-9d7d-4743-95a3-ddd3a2223888 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.659550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f764f-9d7d-4743-95a3-ddd3a2223888 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.659984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f764f-9d7d-4743-95a3-ddd3a2223888 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.662353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 827/1649] 10.222.0.20 () {54 vars in 962 bytes} [Wed Aug 31 11:05:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:17.668520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c4d4-82e5-4e88-9029-8584957a6a31 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.669071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c4d4-82e5-4e88-9029-8584957a6a31 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.669491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7c4d4-82e5-4e88-9029-8584957a6a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.701657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-60a7c4d4-82e5-4e88-9029-8584957a6a31 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:17.705130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60a7c4d4-82e5-4e88-9029-8584957a6a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:17.859343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 823/1650] 10.222.0.20 () {58 vars in 1059 bytes} [Wed Aug 31 11:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:17.866424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62891a1e-f165-4ef8-a510-829b53fb048e None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.867342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62891a1e-f165-4ef8-a510-829b53fb048e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.868209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62891a1e-f165-4ef8-a510-829b53fb048e None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.871195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 828/1651] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 11:05:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:05:17.880282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:17.880545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:17.880817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:17.923002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:17.929768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:17.960193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 824/1652] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:05:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:17.989521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62891a1e-f165-4ef8-a510-829b53fb048e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:17.996673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-62891a1e-f165-4ef8-a510-829b53fb048e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.151197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62891a1e-f165-4ef8-a510-829b53fb048e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:18.159680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:18.160410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:18.160983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:18.161637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:18.191009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:18.192828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:18.228090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 829/1653] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:18.330422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:18.332899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.375993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:18.380718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.525167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fe77b-6e05-4d70-a905-b1a71cbe5d9a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:18.530904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:18.531957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:18.532529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:18.533056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] PATH_INFO: `/v3/users/ff2561c82e5c4015ae0bd1cf2f09cf1d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:18.534888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2561c82e5c4015ae0bd1cf2f09cf1d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:18.537150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:18.565964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8aefc-be38-4f01-a5a8-1a5dd3e9960b service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:18.584895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:18.585763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:18.586055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:18.586327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] PATH_INFO: `/v3/projects/56f3b58753d74bf3bd4e394c2e4a6868` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:18.588082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] RBAC: Authorizing `identity:delete_project(project_id=56f3b58753d74bf3bd4e394c2e4a6868)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:18.590051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:18.622906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2561c82e5c4015ae0bd1cf2f09cf1d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.624735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2561c82e5c4015ae0bd1cf2f09cf1d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.643104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2561c82e5c4015ae0bd1cf2f09cf1d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.651013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:18.679183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2561c82e5c4015ae0bd1cf2f09cf1d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.697076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4cd126c9-6bd2-43bc-8270-9b41957466a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2561c82e5c4015ae0bd1cf2f09cf1d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.721340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 825/1654] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:18 2022] DELETE /identity/v3/users/ff2561c82e5c4015ae0bd1cf2f09cf1d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:18.727415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56f3b58753d74bf3bd4e394c2e4a6868', 'request_id': 'req-9b878bc7-d791-49de-9626-3883b58e635e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.730476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:18.730770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:18.737713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:18.746720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56f3b58753d74bf3bd4e394c2e4a6868', 'request_id': 'req-9b878bc7-d791-49de-9626-3883b58e635e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.766678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56f3b58753d74bf3bd4e394c2e4a6868', 'request_id': 'req-9b878bc7-d791-49de-9626-3883b58e635e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.787534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:18.807860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:18.810349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.812539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] The token cache is being invalidate because project 56f3b58753d74bf3bd4e394c2e4a6868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:18.813131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.816772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:18.824347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 830/1655] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:18 2022] DELETE /identity/v3/projects/56f3b58753d74bf3bd4e394c2e4a6868 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:18.827208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:18.830857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:18.934049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 826/1656] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:18.992931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b878bc7-d791-49de-9626-3883b58e635e admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.009722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.009869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:19.011090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.011731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.012285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] PATH_INFO: `/v3/users/c3e7c2fd7501436abaf63c8978124fb4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.012891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:19.015145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e7c2fd7501436abaf63c8978124fb4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.022387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.146422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e7c2fd7501436abaf63c8978124fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.159523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e7c2fd7501436abaf63c8978124fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.175211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e7c2fd7501436abaf63c8978124fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.200959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e7c2fd7501436abaf63c8978124fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.203207 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e7c2fd7501436abaf63c8978124fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:19.205749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fbfc3-d531-44e6-9965-e72d071f02b4 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:19.214851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 831/1657] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:18 2022] DELETE /identity/v3/users/c3e7c2fd7501436abaf63c8978124fb4 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:19.225664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:19.226321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:19.226604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:19.227303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:19.232835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:19.259165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:19.260695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83cb3c-e4ba-4164-98c7-52b4c59c4e12 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:19.279864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 827/1658] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:21.263355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:21.265917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:21.463373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122386e-1876-442a-8cf8-ae2786439e4f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:21.475357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:21.476354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.476803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.477235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] PATH_INFO: `/v3/projects/f15821c9ec314eda9af3cc72fc281496` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.480626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15821c9ec314eda9af3cc72fc281496)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:21.482818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:21.518651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:21.596893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15821c9ec314eda9af3cc72fc281496', 'request_id': 'req-889825f1-3a0c-4ec7-9da4-99387bd4797a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:21.605155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15821c9ec314eda9af3cc72fc281496', 'request_id': 'req-889825f1-3a0c-4ec7-9da4-99387bd4797a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:21.613599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15821c9ec314eda9af3cc72fc281496', 'request_id': 'req-889825f1-3a0c-4ec7-9da4-99387bd4797a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:21.658662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] The token cache is being invalidate because project f15821c9ec314eda9af3cc72fc281496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:21.659321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:21.662349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-889825f1-3a0c-4ec7-9da4-99387bd4797a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:21.669519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 832/1659] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:21 2022] DELETE /identity/v3/projects/f15821c9ec314eda9af3cc72fc281496 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:21.671094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b667703-16a9-406a-849e-2b887bc1e11c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.673056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b667703-16a9-406a-849e-2b887bc1e11c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.674520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b667703-16a9-406a-849e-2b887bc1e11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.764269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9b667703-16a9-406a-849e-2b887bc1e11c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:21.765888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.768696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b667703-16a9-406a-849e-2b887bc1e11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:21.787111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.787608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:21.896036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] MFA Rules not processed for user `00ff5fd2e6ee4621bf9c9158d2724071`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:21.897498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:21.969270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 828/1660] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:21.972292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:21.972690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:21.973110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.014942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:22.020479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:22.036626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 829/1661] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:22.037747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:22.058880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb055fb-f6cb-40b8-bb30-b907487a2a2d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:22.078250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:22.078250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:22.078250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:22.078250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.082861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:22.084698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:22.091174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:22.120039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:22.136135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 830/1662] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:22.143091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:22.156629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 833/1663] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:21 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:22.165704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28a751-2fbe-4d1c-a3a5-b5309a703512 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:22.167098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:22.168858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:22.185101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:22.185845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:22.186916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:22.187325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.189099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:22.200203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:22.209830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566f4329-6dfc-4634-8a15-99f355aa78ad None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:22.234245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:22.235683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:22.236275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:22.241942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] PATH_INFO: `/v3/projects/26f5f9d4c25e4db3aa3d08b10a247f11` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.256987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=26f5f9d4c25e4db3aa3d08b10a247f11)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:22.270029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:22.336511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 831/1664] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:22 2022] POST /identity/v3/projects => generated 367 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:22.337719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:22.452046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26f5f9d4c25e4db3aa3d08b10a247f11', 'request_id': 'req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:22.502370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26f5f9d4c25e4db3aa3d08b10a247f11', 'request_id': 'req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:22.535588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26f5f9d4c25e4db3aa3d08b10a247f11', 'request_id': 'req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:22.577848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] The token cache is being invalidate because project 26f5f9d4c25e4db3aa3d08b10a247f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:22.578447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:22.580415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:22.580554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e23d1-8555-43da-9a05-0f6e399bb08b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:22.584845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 834/1665] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:22 2022] DELETE /identity/v3/projects/26f5f9d4c25e4db3aa3d08b10a247f11 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:22.590368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:22.595547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:22.653057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:22.792959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:22.795084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:22.795528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:22.795982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.797097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:22.800481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:22.872622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f51c-fed6-4d55-bafc-51dcd750bd3d admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:22.882064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:22.883195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:22.884212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:22.884897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] PATH_INFO: `/v3/users/bd8a1d38c8fb4f678cc63795820fc31c` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.890299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8a1d38c8fb4f678cc63795820fc31c)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:22.894565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:22.908486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 832/1666] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:22 2022] POST /identity/v3/users => generated 284 bytes in 571 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:22.909503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:22.919328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:22.952398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7e6bb-08b7-40d5-a43d-c2d7a9010c9a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:22.960876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:22.962037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:22.962491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:22.963125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:22.964403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:22.966341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:22.996895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 833/1667] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:23.000551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8a1d38c8fb4f678cc63795820fc31c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.010196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:23.030441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8a1d38c8fb4f678cc63795820fc31c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.043770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d389-ea11-4792-92e5-75e2431fb987 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:23.054797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8a1d38c8fb4f678cc63795820fc31c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.062863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:23.064099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.064581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.065019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.065706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:23.067378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:23.092575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8a1d38c8fb4f678cc63795820fc31c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.094446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d76915ce-29a8-412c-9d36-f410e2781e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8a1d38c8fb4f678cc63795820fc31c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:23.095892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 834/1668] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:23.103478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:23.112678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:23.137537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 835/1669] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:22 2022] DELETE /identity/v3/users/bd8a1d38c8fb4f678cc63795820fc31c => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:23.144739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c206c-b5df-4ec2-931c-1c8fc9f833b3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:23.336764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:23.363557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:23.364911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.365359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.365793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] PATH_INFO: `/v3/projects/823accaa7aae43eab299caa87f6f6fd0/users/f347c7bdf1c2452ab8bb623a48385897/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.401975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] RBAC: Authorizing `identity:create_grant(project_id=823accaa7aae43eab299caa87f6f6fd0, user_id=f347c7bdf1c2452ab8bb623a48385897, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:23.408177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:23.440410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 835/1670] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:23 2022] PUT /identity/v3/projects/823accaa7aae43eab299caa87f6f6fd0/users/f347c7bdf1c2452ab8bb623a48385897/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:23.451423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.452427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.452842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.507833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] MFA Rules not processed for user `f347c7bdf1c2452ab8bb623a48385897`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:23.509526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:23.638119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 836/1671] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:23.643429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:23.841928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9da54-af88-4c6a-ab3e-960ebf7f074f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:23.868689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:23.870083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:23.870549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:23.870980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:23.872414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:23.893539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:23.952362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 836/1672] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:23 2022] POST /identity/v3/projects => generated 367 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:23.956924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:23.979436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d32e94-b39d-41b0-ba34-df7fe3c347b4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.033597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:24.169475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.170284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.170683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.171073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.171844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.175250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.248582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 837/1673] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:23 2022] POST /identity/v3/users => generated 290 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:24.250940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.269523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4917877-cc00-422a-8691-6b077846cd73 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.282727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.284238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.285044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.285044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.286697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.289692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.311613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 837/1674] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:24.315924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.333597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1d24f-eba4-4620-bd91-6f8546723d06 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.342535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.343532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.344188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.346458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] PATH_INFO: `/v3/projects/cbad92a141684b7e8c5ef91f707f63b2/users/d53ac2e3285542d98db16337032331c8/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.385555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbad92a141684b7e8c5ef91f707f63b2, user_id=d53ac2e3285542d98db16337032331c8, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.388665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.415452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 838/1675] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:24 2022] PUT /identity/v3/projects/cbad92a141684b7e8c5ef91f707f63b2/users/d53ac2e3285542d98db16337032331c8/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:24.417013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.438238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5b78d-2895-4467-ab18-85c960014b15 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.625288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcd1fe-d6c9-45b4-9e84-2bc93444a958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.627471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcd1fe-d6c9-45b4-9e84-2bc93444a958 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.627923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcd1fe-d6c9-45b4-9e84-2bc93444a958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.628562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcd1fe-d6c9-45b4-9e84-2bc93444a958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.629936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcd1fe-d6c9-45b4-9e84-2bc93444a958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.633123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcd1fe-d6c9-45b4-9e84-2bc93444a958 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.655718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 838/1676] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:24.660251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:24.686397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5963eb-7f9e-4bb1-8c92-7110351df174 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:24.701388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15034efc-b0df-4380-94b8-d764854bdef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:24.705137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15034efc-b0df-4380-94b8-d764854bdef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.705807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15034efc-b0df-4380-94b8-d764854bdef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.706451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15034efc-b0df-4380-94b8-d764854bdef0 admin admin] PATH_INFO: `/v3/projects/cbad92a141684b7e8c5ef91f707f63b2/users/d53ac2e3285542d98db16337032331c8/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.711850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15034efc-b0df-4380-94b8-d764854bdef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbad92a141684b7e8c5ef91f707f63b2, user_id=d53ac2e3285542d98db16337032331c8, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:24.723797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.724897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.725557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.738306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15034efc-b0df-4380-94b8-d764854bdef0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:24.795759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 839/1677] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:24 2022] PUT /identity/v3/projects/cbad92a141684b7e8c5ef91f707f63b2/users/d53ac2e3285542d98db16337032331c8/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:24.807996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:24.809234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:24.810052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:24.871176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] MFA Rules not processed for user `d0e5dae087184c56b1cb72fb7705d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:24.873254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:24.941049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:24.944647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] MFA Rules not processed for user `d53ac2e3285542d98db16337032331c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:24.946313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.024415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 839/1678] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:25.068042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.083145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.136592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 840/1679] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:25.149746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.152745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.402422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c135bcd-3d68-4cf5-88e0-83371efcf8a4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.417863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.419430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.420308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.421188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.445733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae7777-208b-47d7-a6b9-e05cd892e804 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.465948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.467871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.477173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:25.477765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.478636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.480155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.481428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.489366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 841/1680] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:05:25.528873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.530558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.589677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 840/1681] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:25.608436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:25.610568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:25.624205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae54a86-78b5-4c9e-b017-fcc330455c3f service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:25.629262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94095e40-a982-4d86-8820-2f284eed3190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:25.630580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94095e40-a982-4d86-8820-2f284eed3190 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:25.631113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94095e40-a982-4d86-8820-2f284eed3190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:25.631400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94095e40-a982-4d86-8820-2f284eed3190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:25.804778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94095e40-a982-4d86-8820-2f284eed3190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:25.806252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94095e40-a982-4d86-8820-2f284eed3190 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:25.836672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 842/1682] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:26.633537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:26.635235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:26.817224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2f16c-0c56-41de-b281-637b430879aa service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:26.838632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4c6ef-5345-4acd-bcd5-105e3a794269 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:26.839228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4c6ef-5345-4acd-bcd5-105e3a794269 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:26.839449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4c6ef-5345-4acd-bcd5-105e3a794269 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:26.841154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 843/1683] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:05:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:26.841600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:26.842563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:26.842848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:26.843132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] PATH_INFO: `/v3/users/ba116e33912e46abb0a1e732e05b14b7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:26.846542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba116e33912e46abb0a1e732e05b14b7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:26.848551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:26.961932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba116e33912e46abb0a1e732e05b14b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:26.963534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba116e33912e46abb0a1e732e05b14b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:26.967636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:26.968793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:26.969277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:26.979699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba116e33912e46abb0a1e732e05b14b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:26.995743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:27.006766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba116e33912e46abb0a1e732e05b14b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.008942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:27.011707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.027298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b41115ee-9572-4b16-897e-ec3e4567e60d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba116e33912e46abb0a1e732e05b14b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.051429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 841/1684] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:26 2022] DELETE /identity/v3/users/ba116e33912e46abb0a1e732e05b14b7 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:27.061208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b9454-1d00-48cc-a059-2f5beb673f50 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:27.062264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b9454-1d00-48cc-a059-2f5beb673f50 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:27.062559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b9454-1d00-48cc-a059-2f5beb673f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:27.116721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d32b9454-1d00-48cc-a059-2f5beb673f50 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:27.120906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d32b9454-1d00-48cc-a059-2f5beb673f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.156407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 844/1685] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:05:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:27.157668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:27.160236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.183578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723c0f-4e24-4da1-b185-05f0f6de1dc1 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:27.265846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 842/1686] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:27.268194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:27.268437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:27.268721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:27.294089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:27.295176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:27.295697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:27.296125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] PATH_INFO: `/v3/projects/a3e50084410944d283a7a24709c93e72` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:27.299849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e50084410944d283a7a24709c93e72)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:27.302350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:27.317950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] MFA Rules not processed for user `19643be9952649aba81e61d5d55290c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:27.319207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.435581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:27.456232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 843/1687] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:27.457485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:27.464697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.477985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e50084410944d283a7a24709c93e72', 'request_id': 'req-49d127dc-1f11-4713-8bfc-8427096578d7'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.497951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e50084410944d283a7a24709c93e72', 'request_id': 'req-49d127dc-1f11-4713-8bfc-8427096578d7'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.517723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e50084410944d283a7a24709c93e72', 'request_id': 'req-49d127dc-1f11-4713-8bfc-8427096578d7'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.552134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] The token cache is being invalidate because project a3e50084410944d283a7a24709c93e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:27.552134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.562542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:27.566940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 845/1688] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:27 2022] DELETE /identity/v3/projects/a3e50084410944d283a7a24709c93e72 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:27.567851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:27.577571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.626640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6c50e-146b-4a0d-90d8-66b37b1678b3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:27.723862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:27.806873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:27.807696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:27.807990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:27.813272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:27.823197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:27.993152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d127dc-1f11-4713-8bfc-8427096578d7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:28.021560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:28.023750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.025006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.027053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.035398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.081109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:28.083724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:28.085031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1148e2-2a55-4bb4-a69a-db7fa8d1e340 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:28.089224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02175374-715f-4781-88fc-cbfdb28d0aae service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:28.127560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 844/1689] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:28.135935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 846/1690] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:28.137865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.138020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26b22-c3a3-4fa7-b1ee-273666c89b94 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.138202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.138330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26b22-c3a3-4fa7-b1ee-273666c89b94 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.138524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.138623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d26b22-c3a3-4fa7-b1ee-273666c89b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.212022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-74d26b22-c3a3-4fa7-b1ee-273666c89b94 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:28.213395 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-74d26b22-c3a3-4fa7-b1ee-273666c89b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.220666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] MFA Rules not processed for user `19643be9952649aba81e61d5d55290c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:28.222146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.247306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 847/1691] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:28.262053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.262349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.262626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.325262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:28.334923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.388462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 848/1692] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:28.393833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:28.409322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 845/1693] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:28.451910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:28.454793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.459770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c313db-277c-460f-b8ed-7cdd1d304bf0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:28.484831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:28.486359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.486643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.487720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.490312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:28.490629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:28.494389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8f2133-987e-4e4d-8f41-488de1c39a6b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:28.533845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 849/1694] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:28.538395 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f651-1114-4771-a4fb-c5940f3890a2 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.538684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f651-1114-4771-a4fb-c5940f3890a2 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.540374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f651-1114-4771-a4fb-c5940f3890a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.642638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0b55f651-1114-4771-a4fb-c5940f3890a2 None None] MFA Rules not processed for user `0517d50b405e4bce991bb22eef6076a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:28.648713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0b55f651-1114-4771-a4fb-c5940f3890a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.737725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5b1bb-7b14-48f2-866f-5858fadf8985 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:28.759845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:28.760602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.760895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.761154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.767237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:28.816274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:28.827646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:28.892267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 846/1695] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:05:28.895230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:28.921129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58684cd0-0910-4372-a1e0-44bc3277ed06 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:28.931285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:28.933349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.934741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.936255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:28.937815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:28.948162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:28.956798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:28.975310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 850/1696] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:28.988710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:28.988945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:28.989131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.014284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 847/1697] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:28 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:29.020080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:29.048545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] MFA Rules not processed for user `0517d50b405e4bce991bb22eef6076a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:29.050509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:29.085994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:29.176396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d75b23-f9ef-456c-8144-a1a7430fe2c3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:29.201823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:29.205844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:29.206223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:29.206719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.208427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:29.210447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 851/1698] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:29.211222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:29.222981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:29.228367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:29.253035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 848/1699] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:29 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:29.256887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:29.277871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a3f8c9-74d3-482f-b675-aa0adeb53edd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:29.287246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:29.288033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:29.288416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:29.288919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.290499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:29.296603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:29.328745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 849/1700] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:29.332831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:29.353968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df57edf-d7f6-401f-b15b-5c719b1403aa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:29.365439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:29.367921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:29.368351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:29.368730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.371053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:29.376526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:29.398019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f353a5-e1e4-4afc-b7d3-c40f683e0630 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:29.407807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420eab1-6a53-420c-845c-6a195a4cfd68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:29.408794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420eab1-6a53-420c-845c-6a195a4cfd68 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:29.409280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420eab1-6a53-420c-845c-6a195a4cfd68 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:29.409704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420eab1-6a53-420c-845c-6a195a4cfd68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.413737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 850/1701] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:29.415243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:29.425729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420eab1-6a53-420c-845c-6a195a4cfd68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:29.428305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d420eab1-6a53-420c-845c-6a195a4cfd68 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:29.442812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca19ed-5e67-4655-abbc-c2d623eeabd1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:29.457455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 852/1702] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:29.458716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:29.460000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:29.460000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:29.460547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] PATH_INFO: `/v3/projects/44b50f7f739c402fa4275128d1e81bbb/users/4f5868c219334eebacb2c1efa3ccdd0b/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.505136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b50f7f739c402fa4275128d1e81bbb, user_id=4f5868c219334eebacb2c1efa3ccdd0b, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:29.511806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:29.549376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 851/1703] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:29 2022] PUT /identity/v3/projects/44b50f7f739c402fa4275128d1e81bbb/users/4f5868c219334eebacb2c1efa3ccdd0b/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:29.560260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:29.560573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:29.560993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:29.657586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] MFA Rules not processed for user `4f5868c219334eebacb2c1efa3ccdd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:29.662858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:29.882486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 853/1704] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:29.883794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:30.121672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8abaa3-566d-4320-9a31-a26e26579e47 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:30.136046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:30.146474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.146867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.147171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:30.147869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:30.158752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:30.221677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 852/1705] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:29 2022] POST /identity/v3/projects => generated 385 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:30.225770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:30.251530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990413d8-ef97-4bc0-9f43-35c5ecfc5a11 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:30.343351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:30.414714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:30.418079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.418867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.420064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:30.423916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:30.432466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:30.514612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 854/1706] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:30 2022] POST /identity/v3/users => generated 299 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:30.514701 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:30.545315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07f2d01-ff28-4d50-bc86-460500908ff8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:30.561142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:30.561790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.562069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.562338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:30.563566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:30.565264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:30.595793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 853/1707] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:30.600264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:30.613997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5a225-e40b-4f92-b1a3-dd090f67c6e6 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:30.619818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:30.620815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.621243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.621806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] PATH_INFO: `/v3/projects/5106aa5af274466cb7885c40615aeaaa/users/c161fdaf154e421d8465d3cf7d7b7724/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:30.644162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5106aa5af274466cb7885c40615aeaaa, user_id=c161fdaf154e421d8465d3cf7d7b7724, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:30.647015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:30.676194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 855/1708] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:30 2022] PUT /identity/v3/projects/5106aa5af274466cb7885c40615aeaaa/users/c161fdaf154e421d8465d3cf7d7b7724/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:30.681862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:30.690140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:30.713613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4150d61-51f2-4d49-b660-1f59ee62e65c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:30.866557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70236fe-9421-4d9a-80a4-6cdbb1bca76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:30.867600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70236fe-9421-4d9a-80a4-6cdbb1bca76b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.868141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70236fe-9421-4d9a-80a4-6cdbb1bca76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.869136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70236fe-9421-4d9a-80a4-6cdbb1bca76b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:30.870225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70236fe-9421-4d9a-80a4-6cdbb1bca76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:30.873120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70236fe-9421-4d9a-80a4-6cdbb1bca76b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:30.895898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 854/1709] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:05:30.899915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:30.912279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce03783-75d8-4db5-af7b-768e6f39545c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:30.930619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bfd7ec-d44f-4a4e-97d9-bf26c4eb6cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:30.931366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfd7ec-d44f-4a4e-97d9-bf26c4eb6cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.931656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfd7ec-d44f-4a4e-97d9-bf26c4eb6cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.931941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bfd7ec-d44f-4a4e-97d9-bf26c4eb6cf4 admin admin] PATH_INFO: `/v3/projects/5106aa5af274466cb7885c40615aeaaa/users/c161fdaf154e421d8465d3cf7d7b7724/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:30.937564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfd7ec-d44f-4a4e-97d9-bf26c4eb6cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5106aa5af274466cb7885c40615aeaaa, user_id=c161fdaf154e421d8465d3cf7d7b7724, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:30.941011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bfd7ec-d44f-4a4e-97d9-bf26c4eb6cf4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:30.982297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 856/1710] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:30 2022] PUT /identity/v3/projects/5106aa5af274466cb7885c40615aeaaa/users/c161fdaf154e421d8465d3cf7d7b7724/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:30.984436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56424e21-1720-4724-bb30-e12f835c631f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:30.986692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56424e21-1720-4724-bb30-e12f835c631f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:30.987306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56424e21-1720-4724-bb30-e12f835c631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.067496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-56424e21-1720-4724-bb30-e12f835c631f None None] MFA Rules not processed for user `c161fdaf154e421d8465d3cf7d7b7724`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:31.069433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-56424e21-1720-4724-bb30-e12f835c631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.073060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bbd7-4382-4832-b071-5547effafafc None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.073354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bbd7-4382-4832-b071-5547effafafc None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.073620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162bbd7-4382-4832-b071-5547effafafc None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.075254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 857/1711] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:05:31.154133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08196017-721e-49e4-a150-8162e25d9c62 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.154133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08196017-721e-49e4-a150-8162e25d9c62 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.154533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08196017-721e-49e4-a150-8162e25d9c62 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.156391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 858/1712] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 11:05:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:31.189078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.189845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.194119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.212196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 855/1713] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:31.216057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.217189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.217814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.253136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:31.255711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.278551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:31.282567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.399800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 856/1714] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:31.410770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:31.411737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.445269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:31.455360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 859/1715] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:31.456398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:31.457780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.559955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744883e-1060-4e7b-94fe-5be4bf3cc2b1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:31.566752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:31.568351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.568980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.569472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] PATH_INFO: `/v3/users/f347c7bdf1c2452ab8bb623a48385897` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.571715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f347c7bdf1c2452ab8bb623a48385897)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:31.575699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:31.605793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b842ed-68d6-4967-a89f-6b1156ec3e55 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:31.615126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:31.616311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.616807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.617295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.651709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:31.658817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:31.683051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 860/1716] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:31.684091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:31.686238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.706436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe1adc-ea80-4525-a00f-b68960f98672 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:31.714845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:31.715850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.716309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.717211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] PATH_INFO: `/v3/projects/7218420157c142878e694a8379e0113f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.720709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7218420157c142878e694a8379e0113f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:31.723246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:31.724155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f347c7bdf1c2452ab8bb623a48385897'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.725002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f347c7bdf1c2452ab8bb623a48385897'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.748468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f347c7bdf1c2452ab8bb623a48385897'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.778879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f347c7bdf1c2452ab8bb623a48385897'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.798880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f526e92a-15ab-4d57-9096-6a2fdbe99064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f347c7bdf1c2452ab8bb623a48385897'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.812422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 857/1717] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:31 2022] DELETE /identity/v3/users/f347c7bdf1c2452ab8bb623a48385897 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:31.814362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b4716-e630-4104-ab1c-fb4cd4f59c21 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:31.814661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b4716-e630-4104-ab1c-fb4cd4f59c21 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:31.814939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b4716-e630-4104-ab1c-fb4cd4f59c21 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:31.821841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 858/1718] 10.222.0.20 () {54 vars in 912 bytes} [Wed Aug 31 11:05:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:31.822841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b4716-e630-4104-ab1c-fb4cd4f59c21 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:31.827846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:31.843354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d63b4716-e630-4104-ab1c-fb4cd4f59c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:31.911099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7218420157c142878e694a8379e0113f', 'request_id': 'req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.950883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7218420157c142878e694a8379e0113f', 'request_id': 'req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:31.983658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7218420157c142878e694a8379e0113f', 'request_id': 'req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.023054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] The token cache is being invalidate because project 7218420157c142878e694a8379e0113f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:32.023358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.031301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.038192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 861/1719] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:31 2022] DELETE /identity/v3/projects/7218420157c142878e694a8379e0113f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:32.039734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:32.044833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.184894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b4716-e630-4104-ab1c-fb4cd4f59c21 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:32.195047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fc5c8d-00b6-4964-b5f2-4f8704c0fce3 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:32.198074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:32.198973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.199373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.199760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] PATH_INFO: `/v3/users/80361bb4875b4bb8901a0ed5b945eb1f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.201448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80361bb4875b4bb8901a0ed5b945eb1f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:32.205004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:32.212078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:32.213082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.213521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.213949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.219902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.246069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:32.252256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:32.278747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 862/1720] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:32.280504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:32.286744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.294984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:32.310333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80361bb4875b4bb8901a0ed5b945eb1f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.311576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80361bb4875b4bb8901a0ed5b945eb1f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.339456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80361bb4875b4bb8901a0ed5b945eb1f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.359124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80361bb4875b4bb8901a0ed5b945eb1f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.379103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dc075b68-1fa1-44cf-8a2c-d939f7fb85a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80361bb4875b4bb8901a0ed5b945eb1f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.398454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 859/1721] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:31 2022] DELETE /identity/v3/users/80361bb4875b4bb8901a0ed5b945eb1f => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:32.409771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.410054 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.410451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.439812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f534dd0-5953-4b13-9e5a-d8d3249b995a service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:32.459253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:32.460558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.575871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:32.579285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.579618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.579920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.586562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.607821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:32.610241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e3b7e-4427-467a-a6d9-dba64c32c7fd service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:32.616896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:32.623371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 863/1722] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:32.626043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b8f8-bc0b-4d34-af90-f8757607be3d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.633029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b8f8-bc0b-4d34-af90-f8757607be3d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.634628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4b8f8-bc0b-4d34-af90-f8757607be3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.638815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 860/1723] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:32.643810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:32.650157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.680230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0daba-2860-4816-9ca2-f256177e909d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:32.688305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-83e4b8f8-bc0b-4d34-af90-f8757607be3d None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:32.690041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:32.690722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-83e4b8f8-bc0b-4d34-af90-f8757607be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.691655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.692695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.694771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] PATH_INFO: `/v3/users/4f5868c219334eebacb2c1efa3ccdd0b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.697134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f5868c219334eebacb2c1efa3ccdd0b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:32.700925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:32.718528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 864/1724] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:32.720689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.721093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.721472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.753528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:32.754295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.768446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 865/1725] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:32.769491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:32.772071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.789528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cac2f5-1122-4e93-b3e4-d9c89ce9b9da None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:32.794701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:32.795410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.795956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.796401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.821317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:32.822961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:32.851398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 866/1726] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:32.853600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:32.872461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f5868c219334eebacb2c1efa3ccdd0b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.874418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f5868c219334eebacb2c1efa3ccdd0b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.879734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f937b4-76ad-4a95-8024-ccba90a48a08 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:32.885334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d60679-c669-4a8b-8f2c-4034e1853403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:32.886861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d60679-c669-4a8b-8f2c-4034e1853403 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.886861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d60679-c669-4a8b-8f2c-4034e1853403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.886861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d60679-c669-4a8b-8f2c-4034e1853403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.892913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f5868c219334eebacb2c1efa3ccdd0b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.907053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d60679-c669-4a8b-8f2c-4034e1853403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:32.910084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d60679-c669-4a8b-8f2c-4034e1853403 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:32.914027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f5868c219334eebacb2c1efa3ccdd0b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.930046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 867/1727] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:32.930669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b24d7dba-dfb1-4245-b754-93346d8e34fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f5868c219334eebacb2c1efa3ccdd0b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:32.932815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.933404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.933866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.948033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 861/1728] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:32 2022] DELETE /identity/v3/users/4f5868c219334eebacb2c1efa3ccdd0b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:32.950203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:32.950497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:32.950783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:32.982066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:32.984800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:32.989024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:32.994471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:33.060820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:33.108599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 868/1729] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:33.121979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.137050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40631187-bd58-4f8b-adcd-8c4860f63f23 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.148907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.150155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.150441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.152749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.160414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 862/1730] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:33.161635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:33.162053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.163930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.183601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 869/1731] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:33.186581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.199804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125cb0c-580d-43c2-b332-efddcd8ce802 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.207166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.207807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.208156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.208474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.209740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.212308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.282498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 863/1732] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:33 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:33.284136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.314771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5942a-e327-4f35-a683-11cd2dee0a12 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.466973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.469751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:33.510642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.511275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.511552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.511823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.528408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.530371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.617658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 870/1733] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:33 2022] POST /identity/v3/users => generated 282 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:33.628045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.657976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e656f900-c7b8-4f6d-8547-682f4714daba admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.689029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.695463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.695463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.695463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.695463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.711461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.712332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105e89a-6b88-4629-8d05-529e413bb7b6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.729639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 871/1734] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:33.740991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da7ef7-6e12-4dc6-a02a-e4bec7b4599c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.740991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da7ef7-6e12-4dc6-a02a-e4bec7b4599c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.740991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da7ef7-6e12-4dc6-a02a-e4bec7b4599c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.740991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da7ef7-6e12-4dc6-a02a-e4bec7b4599c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.744806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.758058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da7ef7-6e12-4dc6-a02a-e4bec7b4599c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.759073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb591c9-d746-4c20-b991-3b7e243ecd56 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.759528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da7ef7-6e12-4dc6-a02a-e4bec7b4599c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.766973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.768323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.768848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.770134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.771092 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.773046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.790269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 864/1735] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:33.796704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.797631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.805421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.807110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 872/1736] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:33.813484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:33.834758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d726242-9b82-41e7-a353-14dd90dc8886 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:33.846274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca5919-5d43-4224-8e8a-cee9da9f9629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:33.848961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca5919-5d43-4224-8e8a-cee9da9f9629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:33.849543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca5919-5d43-4224-8e8a-cee9da9f9629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:33.849937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca5919-5d43-4224-8e8a-cee9da9f9629 admin admin] PATH_INFO: `/v3/projects/62b096a1c61b4a28b8fadf9fa1b2955e/users/132c9dff0a0e4ed0801729d4cc754abf/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:33.857324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] MFA Rules not processed for user `d0e5dae087184c56b1cb72fb7705d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:33.860236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:33.898196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:33.903716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca5919-5d43-4224-8e8a-cee9da9f9629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62b096a1c61b4a28b8fadf9fa1b2955e, user_id=132c9dff0a0e4ed0801729d4cc754abf, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:33.924815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca5919-5d43-4224-8e8a-cee9da9f9629 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:33.992065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 873/1737] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:33 2022] PUT /identity/v3/projects/62b096a1c61b4a28b8fadf9fa1b2955e/users/132c9dff0a0e4ed0801729d4cc754abf/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:34.007987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:34.008344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:34.012127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:34.073333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 865/1738] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:34.101167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] MFA Rules not processed for user `132c9dff0a0e4ed0801729d4cc754abf`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:34.102667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:34.350300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:34.350902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 874/1739] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:34.373862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014fccb-b40c-4c06-afb2-13e085179e78 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:34.535314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:34.547319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:34.547319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:34.548628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:34.550281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:34.563233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:34.563842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:34.578247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:34.587748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:34.603126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5047c16f-a2db-40c5-87eb-59456cdf6e82 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:34.626915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:34.627836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:34.629036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:34.629182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] PATH_INFO: `/v3/projects/9f40536d651748db8bbb8d8630ab5221` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:34.634654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f40536d651748db8bbb8d8630ab5221)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:34.638015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:34.694498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 866/1740] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:34 2022] POST /identity/v3/projects => generated 365 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:34.699127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:34.840546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f40536d651748db8bbb8d8630ab5221', 'request_id': 'req-72a96925-01e5-4037-a149-04a42ac38569'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:34.859932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f40536d651748db8bbb8d8630ab5221', 'request_id': 'req-72a96925-01e5-4037-a149-04a42ac38569'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:34.880087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f40536d651748db8bbb8d8630ab5221', 'request_id': 'req-72a96925-01e5-4037-a149-04a42ac38569'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:34.902948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] The token cache is being invalidate because project 9f40536d651748db8bbb8d8630ab5221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:34.903315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:34.904817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:34.914778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 875/1741] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:34 2022] DELETE /identity/v3/projects/9f40536d651748db8bbb8d8630ab5221 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:05:34.915809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:34.920073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:34.928366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887be95-69fe-4132-bf42-f4c8affec8f6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:35.075378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:35.076352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.076607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.076951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.077668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:35.079741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:35.131950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 867/1742] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:34 2022] POST /identity/v3/users => generated 289 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:35.131950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:35.136252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:35.147162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a96925-01e5-4037-a149-04a42ac38569 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:35.157715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6f19f-b452-4132-b553-9e5809295965 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:35.162756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:35.163751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.164556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.165296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.167694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:35.171302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:35.172553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.172844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.173118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] PATH_INFO: `/v3/projects/823accaa7aae43eab299caa87f6f6fd0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.174459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=823accaa7aae43eab299caa87f6f6fd0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:35.175508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: WARNING py.warnings [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 11:05:35.175508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: warnings.warn(msg) Aug 31 11:05:35.175508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: Aug 31 11:05:35.177020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:35.187112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:35.195513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:35.198073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:35.237662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 876/1743] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:35.239420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:35.240950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:35.250879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823accaa7aae43eab299caa87f6f6fd0', 'request_id': 'req-4867c15f-d745-4973-8a52-2f4604acfbad'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.261289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c27c70-8ba6-40e9-829f-bfd72f76c9ee service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:35.272560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823accaa7aae43eab299caa87f6f6fd0', 'request_id': 'req-4867c15f-d745-4973-8a52-2f4604acfbad'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.300940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823accaa7aae43eab299caa87f6f6fd0', 'request_id': 'req-4867c15f-d745-4973-8a52-2f4604acfbad'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.346129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] The token cache is being invalidate because project 823accaa7aae43eab299caa87f6f6fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:35.346438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.350361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4867c15f-d745-4973-8a52-2f4604acfbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.368709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 868/1744] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:35 2022] DELETE /identity/v3/projects/823accaa7aae43eab299caa87f6f6fd0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:35.372486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc909c0-6866-4f6a-973b-be4df80c58e7 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.373972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc909c0-6866-4f6a-973b-be4df80c58e7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.380253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc909c0-6866-4f6a-973b-be4df80c58e7 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.387266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 869/1745] 10.222.0.20 () {54 vars in 961 bytes} [Wed Aug 31 11:05:35 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:35.388626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc909c0-6866-4f6a-973b-be4df80c58e7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:35.395667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5fc909c0-6866-4f6a-973b-be4df80c58e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:35.404929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:35.493197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:35.496471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.496471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.497036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] PATH_INFO: `/v3/users/00ff5fd2e6ee4621bf9c9158d2724071` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.503978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ff5fd2e6ee4621bf9c9158d2724071)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:35.521983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:35.527843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc909c0-6866-4f6a-973b-be4df80c58e7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:35.543036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:35.543681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.543962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.544283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.544950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:35.546613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:35.583264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 870/1746] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:35.584588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:35.591331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:35.635030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b78f2-a574-45d5-bbf3-6138a280f5c2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:35.643892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:35.645779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.646257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.646857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] PATH_INFO: `/v3/users/d53ac2e3285542d98db16337032331c8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.653169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53ac2e3285542d98db16337032331c8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:35.657042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:35.660194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ff5fd2e6ee4621bf9c9158d2724071'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.682136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ff5fd2e6ee4621bf9c9158d2724071'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.702251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ff5fd2e6ee4621bf9c9158d2724071'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.731510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ff5fd2e6ee4621bf9c9158d2724071'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.733635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ffae8bef-c08c-4b50-b591-ab2f1b934a7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ff5fd2e6ee4621bf9c9158d2724071'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.752788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 877/1747] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:35 2022] DELETE /identity/v3/users/00ff5fd2e6ee4621bf9c9158d2724071 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:35.755732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.757288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.757691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.831126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] MFA Rules not processed for user `19643be9952649aba81e61d5d55290c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:35.832474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:35.839694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53ac2e3285542d98db16337032331c8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.843085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53ac2e3285542d98db16337032331c8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.866351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53ac2e3285542d98db16337032331c8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.890427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53ac2e3285542d98db16337032331c8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.899403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e92a0d2-e047-46b2-8d0e-181daeeb7679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53ac2e3285542d98db16337032331c8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:35.911960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 871/1748] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:35 2022] DELETE /identity/v3/users/d53ac2e3285542d98db16337032331c8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:35.911960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:35.911960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:35.911960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:35.939915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] MFA Rules not processed for user `f5103a11f8f446778043577e13cea822`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:35.941298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:36.050698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 878/1749] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:36.052207 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:36.059567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:36.088788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 872/1750] 10.222.0.20 () {58 vars in 1058 bytes} [Wed Aug 31 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:36.092544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:36.100234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:36.124286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:36.165120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:36.326107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e8af-dd81-4d40-9da2-c0d540db318c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:36.428619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e73dd-4239-4234-aba5-ce2a86eb55c0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:36.441542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:36.442405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:36.443014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:36.443448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] PATH_INFO: `/v3/projects/44b50f7f739c402fa4275128d1e81bbb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:36.445261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b50f7f739c402fa4275128d1e81bbb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:36.451560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:36.459635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:36.460711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:36.461190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:36.461545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] PATH_INFO: `/v3/projects/71692d19f9cb4f11935e9f9c6f81bba5/users/1b38c49f65d14200ad0962ef78d48bbe/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:36.490834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71692d19f9cb4f11935e9f9c6f81bba5, user_id=1b38c49f65d14200ad0962ef78d48bbe, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:36.498518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:36.530771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 879/1751] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:36 2022] PUT /identity/v3/projects/71692d19f9cb4f11935e9f9c6f81bba5/users/1b38c49f65d14200ad0962ef78d48bbe/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:36.533015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:36.558302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b50f7f739c402fa4275128d1e81bbb', 'request_id': 'req-8bf00382-fdfc-4723-91f6-fec6193c3e12'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:36.574249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b50f7f739c402fa4275128d1e81bbb', 'request_id': 'req-8bf00382-fdfc-4723-91f6-fec6193c3e12'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:36.618162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b50f7f739c402fa4275128d1e81bbb', 'request_id': 'req-8bf00382-fdfc-4723-91f6-fec6193c3e12'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:36.649744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] The token cache is being invalidate because project 44b50f7f739c402fa4275128d1e81bbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:36.651044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:36.651248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:36.657050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 873/1752] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:36 2022] DELETE /identity/v3/projects/44b50f7f739c402fa4275128d1e81bbb => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:36.657050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:36.657647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:36.800469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0cc3c-0d58-452b-9f22-7b7af125f586 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:36.842122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf00382-fdfc-4723-91f6-fec6193c3e12 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:36.861956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:36.863725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:36.865020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:36.867045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:36.867985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:36.873525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:36.911909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 874/1753] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:36.916808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:36.923488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:36.956338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedfc099-87aa-458d-a91b-152ab96c61a5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:36.972604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:36.973566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:36.975441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:36.976436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:36.977866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] PATH_INFO: `/v3/users/c161fdaf154e421d8465d3cf7d7b7724` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:36.982585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c161fdaf154e421d8465d3cf7d7b7724)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:36.993938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:37.016647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:37.017479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:37.018224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:37.019114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:37.022640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.043378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:37.046204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:37.061694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 880/1754] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:37.063279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:37.065084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.081603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86c121-d25a-4ddf-96ed-79f76e8f4128 service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:37.092049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:37.093119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:37.094260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:37.094764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] PATH_INFO: `/v3/projects/5afe8fdbf8764e96a7dee56746c08c1e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:37.097857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afe8fdbf8764e96a7dee56746c08c1e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:37.103157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:37.116380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:37.178470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c161fdaf154e421d8465d3cf7d7b7724'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.179885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c161fdaf154e421d8465d3cf7d7b7724'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.214572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c161fdaf154e421d8465d3cf7d7b7724'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.266465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c161fdaf154e421d8465d3cf7d7b7724'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.284560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c161fdaf154e421d8465d3cf7d7b7724'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.292451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afe8fdbf8764e96a7dee56746c08c1e', 'request_id': 'req-87310bb7-e4e0-4297-9d27-e087a086fe9e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.308856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 875/1755] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:36 2022] DELETE /identity/v3/users/c161fdaf154e421d8465d3cf7d7b7724 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:37.309818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afe8fdbf8764e96a7dee56746c08c1e', 'request_id': 'req-87310bb7-e4e0-4297-9d27-e087a086fe9e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.310067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:37.315166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.326293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afe8fdbf8764e96a7dee56746c08c1e', 'request_id': 'req-87310bb7-e4e0-4297-9d27-e087a086fe9e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.363819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] The token cache is being invalidate because project 5afe8fdbf8764e96a7dee56746c08c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:37.364618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.373684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-87310bb7-e4e0-4297-9d27-e087a086fe9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:37.388329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 881/1756] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:37 2022] DELETE /identity/v3/projects/5afe8fdbf8764e96a7dee56746c08c1e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:37.403686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badaed1-3a91-4770-9ddd-9d96df27aa0c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:37.403983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badaed1-3a91-4770-9ddd-9d96df27aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:37.404297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1badaed1-3a91-4770-9ddd-9d96df27aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:37.530383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1badaed1-3a91-4770-9ddd-9d96df27aa0c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:37.532378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1badaed1-3a91-4770-9ddd-9d96df27aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.535784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95240604-4ee3-4c17-af63-ad471636054c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:37.551806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1834fe-a0d2-4761-94ac-297cc4fdd537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:37.554588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1834fe-a0d2-4761-94ac-297cc4fdd537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:37.555155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1834fe-a0d2-4761-94ac-297cc4fdd537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:37.555692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1834fe-a0d2-4761-94ac-297cc4fdd537 admin admin] PATH_INFO: `/v3/projects/71692d19f9cb4f11935e9f9c6f81bba5/users/1b38c49f65d14200ad0962ef78d48bbe/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:37.558641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1834fe-a0d2-4761-94ac-297cc4fdd537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71692d19f9cb4f11935e9f9c6f81bba5, user_id=1b38c49f65d14200ad0962ef78d48bbe, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:37.564844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1834fe-a0d2-4761-94ac-297cc4fdd537 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:37.584318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 882/1757] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:37.585577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 876/1758] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:37 2022] PUT /identity/v3/projects/71692d19f9cb4f11935e9f9c6f81bba5/users/1b38c49f65d14200ad0962ef78d48bbe/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:37.591889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:37.592572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:37.595905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:37.597300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:37.597594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:37.597874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:37.626966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:37.628848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.646246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] MFA Rules not processed for user `1b38c49f65d14200ad0962ef78d48bbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:37.651062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.862747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 883/1759] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:37.866940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:37.874752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:37.885247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 877/1760] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:37.888668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:37.975459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329098f-f3b4-4025-9f42-67630bf7a06d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:37.978218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cbc1-1743-4d29-84f7-2b585ebeb69c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:37.996918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:38.003944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:38.004848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:38.006420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:38.009387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:38.010237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:38.019270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:38.023786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:38.032505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:38.036975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:38.039433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] PATH_INFO: `/v3/projects/cbad92a141684b7e8c5ef91f707f63b2` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:38.044233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbad92a141684b7e8c5ef91f707f63b2)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:38.053512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:38.060982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 878/1761] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:38.064647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:38.071352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:38.130487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:38.239023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbad92a141684b7e8c5ef91f707f63b2', 'request_id': 'req-efa73196-b07d-48a0-9e0a-6fb9dd558f56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:38.263922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:38.288063 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbad92a141684b7e8c5ef91f707f63b2', 'request_id': 'req-efa73196-b07d-48a0-9e0a-6fb9dd558f56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:38.302789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6e237-f575-4b42-ba4d-d64a6502f870 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:38.303044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbad92a141684b7e8c5ef91f707f63b2', 'request_id': 'req-efa73196-b07d-48a0-9e0a-6fb9dd558f56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:38.345506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] The token cache is being invalidate because project cbad92a141684b7e8c5ef91f707f63b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:38.346318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:38.348669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-efa73196-b07d-48a0-9e0a-6fb9dd558f56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:38.353025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 884/1762] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:37 2022] DELETE /identity/v3/projects/cbad92a141684b7e8c5ef91f707f63b2 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:38.355060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18855c2-4814-4d53-8a49-d523f85ac1a1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:38.357780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18855c2-4814-4d53-8a49-d523f85ac1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:38.358726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18855c2-4814-4d53-8a49-d523f85ac1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:38.459130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:38.459876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:38.460209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:38.460587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:38.462379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:38.478733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f18855c2-4814-4d53-8a49-d523f85ac1a1 None None] MFA Rules not processed for user `714872990c384663bcd564cd5b9ae432`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:38.485513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f18855c2-4814-4d53-8a49-d523f85ac1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:38.603320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:38.606034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:38.659051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 879/1763] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:38.660059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:38.668743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:38.728775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 885/1764] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:38.732052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeeb7c-60ee-4b49-a913-1de3abc508c8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:38.735459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeeb7c-60ee-4b49-a913-1de3abc508c8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:38.736167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeeb7c-60ee-4b49-a913-1de3abc508c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:38.833340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-edbeeb7c-60ee-4b49-a913-1de3abc508c8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:38.839710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-edbeeb7c-60ee-4b49-a913-1de3abc508c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:39.029441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 886/1765] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:39.038477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:39.039118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:39.039657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:39.075502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e4c778-f956-494c-b1ff-64b94a0d1d46 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:39.101962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:39.103033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:39.114027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:39.114430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:39.118150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:39.120488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:39.123952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:39.132251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:39.145439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 887/1766] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:39.145439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:39.148367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:39.208868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc657d4-c652-486a-83ce-0c78f7f0b20e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:39.233093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 880/1767] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:38 2022] POST /identity/v3/projects => generated 381 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:39.234777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:39.240610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:39.255427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:39.265231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:39.266511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:39.266993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:39.267292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] PATH_INFO: `/v3/projects/5106aa5af274466cb7885c40615aeaaa` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:39.277396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5106aa5af274466cb7885c40615aeaaa)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:39.287935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:39.375398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdd41c6-b7eb-4f49-b216-401ec2556ab1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:39.398985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:39.399238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:39.402631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:39.403027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:39.611889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:39.613594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:39.633492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 881/1768] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:39.634338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:39.693896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5106aa5af274466cb7885c40615aeaaa', 'request_id': 'req-dfa3ea11-20db-42be-9011-dd00204bcb11'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:39.750862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5106aa5af274466cb7885c40615aeaaa', 'request_id': 'req-dfa3ea11-20db-42be-9011-dd00204bcb11'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:39.772928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad71e76-c111-44cb-bc5e-e4653adc2f49 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:39.803190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:39.814539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5106aa5af274466cb7885c40615aeaaa', 'request_id': 'req-dfa3ea11-20db-42be-9011-dd00204bcb11'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:39.897255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] The token cache is being invalidate because project 5106aa5af274466cb7885c40615aeaaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:39.897904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:39.904565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:39.929128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 888/1769] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:39 2022] DELETE /identity/v3/projects/5106aa5af274466cb7885c40615aeaaa => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:39.929128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:39.938957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:39.965360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:40.002024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:40.005529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:40.007280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:40.008165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:40.009666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:40.010677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:40.014688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ff53-f0c4-4b13-8b71-b24a3431bc44 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:40.048590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 882/1770] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:40.051885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:40.053007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:40.054223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:40.254929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ea11-20db-42be-9011-dd00204bcb11 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:40.299136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1f365-4139-4f2a-aecd-294c2981ab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:40.307439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f365-4139-4f2a-aecd-294c2981ab72 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:40.307739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f365-4139-4f2a-aecd-294c2981ab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:40.308125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f365-4139-4f2a-aecd-294c2981ab72 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:40.309333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f365-4139-4f2a-aecd-294c2981ab72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:40.322934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f365-4139-4f2a-aecd-294c2981ab72 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:40.363452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:40.366714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:40.401210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 889/1771] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:39 2022] POST /identity/v3/users => generated 291 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:40.406892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:40.406892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:40.406892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:40.478058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:40.479439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:40.492146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 890/1772] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:40.492146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:40.506697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:40.538437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ee996-d218-40e8-b991-23da30d260e4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:40.558540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:40.559816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:40.562691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:40.565577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:40.569130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:40.580727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:40.688478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 883/1773] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:05:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 638 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:40.693036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:40.700380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:40.736383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 891/1774] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:40 2022] POST /identity/v3/projects => generated 399 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:40.738299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:40.777261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224f9d5-8877-4826-b707-f288d3c0865d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:41.229532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:41.231661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:41.232172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:41.232350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] PATH_INFO: `/v3/users/132c9dff0a0e4ed0801729d4cc754abf` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:41.233873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=132c9dff0a0e4ed0801729d4cc754abf)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:41.236648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:41.265239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553a1ca-625c-4e4f-ad1d-1296939c87c9 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:41.289282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d8695-b412-403c-9a33-50c1a3ec2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:41.290287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8695-b412-403c-9a33-50c1a3ec2079 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:41.290806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8695-b412-403c-9a33-50c1a3ec2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:41.295226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d8695-b412-403c-9a33-50c1a3ec2079 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:41.305537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d8695-b412-403c-9a33-50c1a3ec2079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:41.317237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d8695-b412-403c-9a33-50c1a3ec2079 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:41.376183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 892/1775] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 639 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:41.387618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:41.389460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:41.389998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:41.419874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132c9dff0a0e4ed0801729d4cc754abf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:41.420073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132c9dff0a0e4ed0801729d4cc754abf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:41.435898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132c9dff0a0e4ed0801729d4cc754abf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:41.438045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:41.464254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132c9dff0a0e4ed0801729d4cc754abf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:41.483749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132c9dff0a0e4ed0801729d4cc754abf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:41.499435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 884/1776] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:40 2022] DELETE /identity/v3/users/132c9dff0a0e4ed0801729d4cc754abf => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:41.500531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:41.505473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:41.527144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:41.529733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:41.664527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa3040-5eca-4453-8870-1828756ea524 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:41.670868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:41.683365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:41.686929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:41.688611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:41.693984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:41.699592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:41.726170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:41.779831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 885/1777] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:41 2022] POST /identity/v3/users => generated 300 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:41.789428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:41.795919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:41.815368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 893/1778] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:41.827615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:41.830011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:41.887546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef112f-9ca1-4e05-bb79-101c6bde8f04 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:41.905401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151619b-fe36-495a-ab59-3550894fbb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:41.907216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151619b-fe36-495a-ab59-3550894fbb53 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:41.907481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151619b-fe36-495a-ab59-3550894fbb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:41.907857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151619b-fe36-495a-ab59-3550894fbb53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:41.909006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151619b-fe36-495a-ab59-3550894fbb53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:41.913377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151619b-fe36-495a-ab59-3550894fbb53 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:41.949841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 894/1779] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:41.953642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:41.955009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:41.956180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:42.081713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:42.084965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:42.117408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 895/1780] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:42.120677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:42.146396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557b790-bd5c-4436-aaaf-2211dfeb94a8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:42.161157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:42.164487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:42.164953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:42.165445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:42.167433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:42.169406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:42.202494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 896/1781] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:42.205406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:42.238662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713670d-0dc1-46a3-b799-71e805e76dbf admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:42.252611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:42.254536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:42.256033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:42.256724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:42.257492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:42.272629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:42.279717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:42.316285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85478a59-8d61-42da-92a7-48e1767d53f8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:42.335965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 897/1782] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:42.338020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:42.338807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:42.340033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:42.340571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:42.342833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:42.354800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:42.366283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb04ce2-a9a1-4223-9fb1-8b5d38588087 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:42.377423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62387bcd-cbb1-49bf-b6bc-73cb790733c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:42.379561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387bcd-cbb1-49bf-b6bc-73cb790733c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:42.380262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387bcd-cbb1-49bf-b6bc-73cb790733c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:42.381034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387bcd-cbb1-49bf-b6bc-73cb790733c9 admin admin] PATH_INFO: `/v3/projects/5d43deb553414691a425ef3c79b7843b/users/a9fb4ad3ce1e4250a01fc16af3796153/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:42.423027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62387bcd-cbb1-49bf-b6bc-73cb790733c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d43deb553414691a425ef3c79b7843b, user_id=a9fb4ad3ce1e4250a01fc16af3796153, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:42.429652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62387bcd-cbb1-49bf-b6bc-73cb790733c9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:42.488047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 898/1783] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:42 2022] PUT /identity/v3/projects/5d43deb553414691a425ef3c79b7843b/users/a9fb4ad3ce1e4250a01fc16af3796153/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:42.488047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:42.488047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:42.488047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:42.590572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:42.593765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:42.654825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:42.742548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:42.745691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:42.818441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 886/1784] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:05:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 1032 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:05:42.820131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:42.830763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:42.922237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 899/1785] 10.222.0.55 () {58 vars in 1015 bytes} [Wed Aug 31 11:05:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:42.924876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:43.200884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a665f-a6d6-41d1-8dd6-cd29daa02637 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:43.223836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb264b3-3981-40da-9d0b-17be045c5e24 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:43.228296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cde6aa-a237-4711-b530-ef3923e6ceed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:43.229391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cde6aa-a237-4711-b530-ef3923e6ceed service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:43.229888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cde6aa-a237-4711-b530-ef3923e6ceed service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:43.230359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cde6aa-a237-4711-b530-ef3923e6ceed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:43.271103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:43.272978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:43.274008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:43.274008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:43.281786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:43.286820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:43.295833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cde6aa-a237-4711-b530-ef3923e6ceed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:43.323779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cde6aa-a237-4711-b530-ef3923e6ceed service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:43.339166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 900/1786] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:43.339166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:43.355869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3e2d0-34ce-4832-ac93-56c135583b68 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:43.367596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 887/1787] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:43.370275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:43.370769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:43.371280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:43.374161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:43.376123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:43.376730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:43.380709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:43.384166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:43.391527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:43.461414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 901/1788] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:43 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:43.464287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:43.473093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:43.549554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] MFA Rules not processed for user `a9fb4ad3ce1e4250a01fc16af3796153`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:43.551465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:43.637961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5cd4b-1a10-4806-8338-3f3aafe569f6 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:43.647265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:43.648164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:43.648451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:43.648730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:43.677693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:43.678739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:43.729427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 902/1789] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:43.734871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:43.912442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 888/1790] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 543 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:43.914703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:43.972042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734166f8-2ca5-438f-9a3f-33a7d247c87d service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:43.996343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:43.997371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:43.997761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:43.997884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] PATH_INFO: `/v3/projects/8e609628bade4e388ae76a865dddf570/users/abb5577a789042eb9510e4c8567b4dea/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.063599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e609628bade4e388ae76a865dddf570, user_id=abb5577a789042eb9510e4c8567b4dea, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:44.065818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:44.104531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 903/1791] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:43 2022] PUT /identity/v3/projects/8e609628bade4e388ae76a865dddf570/users/abb5577a789042eb9510e4c8567b4dea/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:44.105129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:44.274596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee7e875-d96f-4e35-af52-ef47c302a686 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:44.291689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737d3e-283c-4287-b608-0fac250edd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:44.292355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737d3e-283c-4287-b608-0fac250edd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:44.292643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737d3e-283c-4287-b608-0fac250edd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:44.292934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737d3e-283c-4287-b608-0fac250edd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.293709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47737d3e-283c-4287-b608-0fac250edd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:44.295628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47737d3e-283c-4287-b608-0fac250edd2f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:44.324266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0dba26-959e-45af-a2ff-c47f9f5781d6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:44.348300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:44.351016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:44.351564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:44.352091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.354595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:44.362410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:44.372621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 904/1792] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:44 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:44.377052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:44.377336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:44.377713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.413073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 889/1793] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:43 2022] POST /identity/v3/users => generated 293 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:44.422049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:44.449121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:44.465866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af235182-119c-4033-9534-46b68fc38e9f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:44.470478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:44.479120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] MFA Rules not processed for user `abb5577a789042eb9510e4c8567b4dea`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:44.486527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:44.789585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 905/1794] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:44 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:44.794289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:44.817278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:44.817599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:44.817758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:44.818526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.818692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:44.837399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:44.842604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3de2-32b5-489e-8b13-20091114dd2e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:44.862591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:44.863631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:44.867729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:44.868376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.869684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:44.878236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:44.942416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 890/1795] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:44 2022] POST /identity/v3/users => generated 297 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:44.943996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:44.954679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 906/1796] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:44.956449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:44.976225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6580e02-662a-4e44-b4ef-fbe558c1342a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:44.988394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:44.989294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:44.989742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:44.990175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:44.990982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:44.994575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.025155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0baf88-7601-417b-9c6e-658c76e6f96b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.044241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.044711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 891/1797] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:44 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:45.047853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.050335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.053460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.055050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.056054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.057996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.072679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 907/1798] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:45.073904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.089438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29e7e0-e372-4f11-9945-d74c50acfbec admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.223793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.224467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.229826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.230266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.231337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.233651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.320721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 908/1799] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:45 2022] POST /identity/v3/users => generated 307 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:45.325806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.353693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdda466-b077-4b9e-a157-7e67f82aac87 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.388787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.393343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.394387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.394851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a858f6-2afa-4994-b024-46b007ff3366 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.395653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] PATH_INFO: `/v3/projects/35a8738ab6bb4fe19292464f79464dd6/users/2b480d8de0e34882905792616751c6f6/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.425001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.434560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.435913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.439317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.442793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.458133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.500098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a8738ab6bb4fe19292464f79464dd6, user_id=2b480d8de0e34882905792616751c6f6, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.512838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.517716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 892/1800] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 472 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:45.518349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.556787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169077ff-8f75-4b58-af75-a7c56f8f8634 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.557406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 909/1801] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:45 2022] PUT /identity/v3/projects/35a8738ab6bb4fe19292464f79464dd6/users/2b480d8de0e34882905792616751c6f6/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:45.563684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.789818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:45.826055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.828444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.828754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.829133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.829706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.831346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.848750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4d445-7231-4d72-ba9d-e8157d0f51d0 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.859687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.860569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.860743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.861001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] PATH_INFO: `/v3/projects/cd08c3763d1f4a1c8f045d0cee00f604/users/d26124332cbf4577ae65bb50a7ea5b45/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.867443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 893/1802] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:45.868516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.883748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:45.897383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8a088-d718-4d75-bd44-76513ddbce33 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:45.904891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd08c3763d1f4a1c8f045d0cee00f604, user_id=d26124332cbf4577ae65bb50a7ea5b45, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.912362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:45.913239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:45.914068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.914507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:45.916026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:45.917320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:45.924096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:45.964522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 910/1803] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:45 2022] PUT /identity/v3/projects/cd08c3763d1f4a1c8f045d0cee00f604/users/d26124332cbf4577ae65bb50a7ea5b45/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:45.968694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:45.972241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 894/1804] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:45 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:45.977493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:46.023881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c10526a-a087-41f4-8b5e-e6ef0eaad47c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:46.258762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669b342-4fa1-46ff-a6b2-1116e3fbdba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:46.260098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669b342-4fa1-46ff-a6b2-1116e3fbdba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.260541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669b342-4fa1-46ff-a6b2-1116e3fbdba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.262466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669b342-4fa1-46ff-a6b2-1116e3fbdba5 admin admin] PATH_INFO: `/v3/projects/35a8738ab6bb4fe19292464f79464dd6/users/2b480d8de0e34882905792616751c6f6/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.274839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669b342-4fa1-46ff-a6b2-1116e3fbdba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a8738ab6bb4fe19292464f79464dd6, user_id=2b480d8de0e34882905792616751c6f6, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:46.282838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669b342-4fa1-46ff-a6b2-1116e3fbdba5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:46.295621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152eff37-29e3-42f8-9a2e-7d70a63635c3 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:46.310677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 895/1805] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:45 2022] PUT /identity/v3/projects/35a8738ab6bb4fe19292464f79464dd6/users/2b480d8de0e34882905792616751c6f6/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:46.316505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bb67a-a9d2-41fb-b7e5-1ac283556977 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.317310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bb67a-a9d2-41fb-b7e5-1ac283556977 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.318111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8bb67a-a9d2-41fb-b7e5-1ac283556977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.394807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5d8bb67a-a9d2-41fb-b7e5-1ac283556977 None None] MFA Rules not processed for user `d26124332cbf4577ae65bb50a7ea5b45`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:46.396839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d8bb67a-a9d2-41fb-b7e5-1ac283556977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:46.553223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:46.554234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.554234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.554548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] PATH_INFO: `/v3/projects/75d28216713146cd9a92d7d10b1c19b4/users/b920e99eecb546b7a08047f853220423/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.619003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 896/1806] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:46.619239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d28216713146cd9a92d7d10b1c19b4, user_id=b920e99eecb546b7a08047f853220423, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:46.619424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:46.619424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:46.619424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:46.619859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:46.688957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 911/1807] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:45 2022] PUT /identity/v3/projects/75d28216713146cd9a92d7d10b1c19b4/users/b920e99eecb546b7a08047f853220423/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:46.688957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:46.693918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:46.712317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] MFA Rules not processed for user `2b480d8de0e34882905792616751c6f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:46.713797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:46.900473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 897/1808] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:46 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:46.901144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:46.907283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:46.973573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64f0a4-6b94-4b77-aa1b-002ca9ce0049 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:47.001560 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:47.002675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:47.003166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:47.008832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:47.060308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:47.237849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0541e8-5aa9-4b00-af1e-2e937a5fc465 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:47.251976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:47.253887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:47.255071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:47.255456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:47.258952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:47.269128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:47.309673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:47.315476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:47.344998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 898/1809] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:47.349732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:47.355394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:47.372375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 912/1810] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:47.376413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:47.400294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699282c-42db-4a5b-b683-f54f405a7edc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:47.431860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e0e4d-5330-4ff9-8515-7418663475c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:47.439056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e03658-9704-4f98-83c3-e3284997a5c4 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:47.442801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e0e4d-5330-4ff9-8515-7418663475c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:47.443134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e0e4d-5330-4ff9-8515-7418663475c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:47.443421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e0e4d-5330-4ff9-8515-7418663475c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:47.461323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:47.465773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:47.475112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:47.477245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] PATH_INFO: `/v3/projects/75d28216713146cd9a92d7d10b1c19b4/users/b920e99eecb546b7a08047f853220423/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:47.488957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e0e4d-5330-4ff9-8515-7418663475c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:47.490663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75d28216713146cd9a92d7d10b1c19b4, user_id=b920e99eecb546b7a08047f853220423, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:47.493341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e0e4d-5330-4ff9-8515-7418663475c1 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:47.521225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:47.558617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 899/1811] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:47 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:47.576069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 913/1812] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:47 2022] PUT /identity/v3/projects/75d28216713146cd9a92d7d10b1c19b4/users/b920e99eecb546b7a08047f853220423/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:47.587353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93056-2480-47df-9adb-50541b60c61b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:47.587702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93056-2480-47df-9adb-50541b60c61b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:47.588138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be93056-2480-47df-9adb-50541b60c61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:47.750148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6be93056-2480-47df-9adb-50541b60c61b None None] MFA Rules not processed for user `b920e99eecb546b7a08047f853220423`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:47.762606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6be93056-2480-47df-9adb-50541b60c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:48.250937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 900/1813] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:47 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 665 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:48.592649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:48.809809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be93056-2480-47df-9adb-50541b60c61b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:48.813003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6be93056-2480-47df-9adb-50541b60c61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:48.899585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789ead0-3759-4cce-9d09-ca98fc491d02 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:48.947558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:48.954757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:48.955329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:48.957170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:48.984291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:48.988267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.006721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be93056-2480-47df-9adb-50541b60c61b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:49.017269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:49.025788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:49.026078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:49.027037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] PATH_INFO: `/v3/projects/62b096a1c61b4a28b8fadf9fa1b2955e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:49.028598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62b096a1c61b4a28b8fadf9fa1b2955e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:49.031026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.064794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:49.076469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 914/1814] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:48 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:49.080452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:49.177107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62b096a1c61b4a28b8fadf9fa1b2955e', 'request_id': 'req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.195910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62b096a1c61b4a28b8fadf9fa1b2955e', 'request_id': 'req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.212668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62b096a1c61b4a28b8fadf9fa1b2955e', 'request_id': 'req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.258840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] The token cache is being invalidate because project 62b096a1c61b4a28b8fadf9fa1b2955e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:49.260282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.261631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:49.267269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 901/1815] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:48 2022] DELETE /identity/v3/projects/62b096a1c61b4a28b8fadf9fa1b2955e => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:49.269872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:49.276691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:49.346739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33420872-e3ab-4f27-809d-19c7b3fb1fbd service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:49.477809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e5183-7764-46e4-ab7b-f1743e1b38b7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:49.487373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:49.489733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:49.489733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:49.490304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:49.494920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:49.514885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:49.539616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b480d8de0e34882905792616751c6f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a8738ab6bb4fe19292464f79464dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:49.543252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:49.545012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:49.545708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:49.548607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:49.549003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:49.553495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.572189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 915/1816] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:49.573462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:49.576156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:49.660885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:49.662994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.690919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 902/1817] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:49 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:49.693218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:49.699716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:49.740959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe616e7-8810-43c2-89ab-8dfbef5c2648 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:49.758471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b480d8de0e34882905792616751c6f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a8738ab6bb4fe19292464f79464dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:49.759457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:49.760465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:49.760983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:49.762217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:49.765568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:49.833480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 903/1818] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:49 2022] POST /identity/v3/projects => generated 354 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:49.837515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:49.872620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c155ad-78f3-4288-a624-0c63718c3bd5 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:50.001780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:50.055971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc4ae8-dfde-4fb7-a889-9f16b2606c5a tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:50.065134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b480d8de0e34882905792616751c6f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a8738ab6bb4fe19292464f79464dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:50.067342 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:50.067705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:50.068087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:50.069065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:50.069563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:50.071708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:50.092511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 904/1819] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:50.163692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:50.164503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:50.164847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:50.165132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] PATH_INFO: `/v3/users/1b38c49f65d14200ad0962ef78d48bbe` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:50.177107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b38c49f65d14200ad0962ef78d48bbe)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:50.186337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:50.402525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b38c49f65d14200ad0962ef78d48bbe'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.437473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b38c49f65d14200ad0962ef78d48bbe'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.477983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b38c49f65d14200ad0962ef78d48bbe'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.514220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b38c49f65d14200ad0962ef78d48bbe'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.516711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6d7f4d0c-3140-43e3-9bc8-c2cccd8bf989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b38c49f65d14200ad0962ef78d48bbe'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:50.534191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 916/1820] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:49 2022] DELETE /identity/v3/users/1b38c49f65d14200ad0962ef78d48bbe => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:50.699521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:50.703888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:50.752571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b4aa0-98b5-40d3-9d63-a898790a6fcf tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:50.931994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2b480d8de0e34882905792616751c6f6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35a8738ab6bb4fe19292464f79464dd6', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:50.932755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:50.933989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:50.935153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] PATH_INFO: `/v3/projects/668309102e644abcb175f1df90a77e50` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:50.939366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorizing `identity:delete_project(project_id=668309102e644abcb175f1df90a77e50)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:50.947321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.087367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '668309102e644abcb175f1df90a77e50', 'request_id': 'req-131c0905-0d90-4d9d-872b-135c88bad9a7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.146340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '668309102e644abcb175f1df90a77e50', 'request_id': 'req-131c0905-0d90-4d9d-872b-135c88bad9a7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.154650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '668309102e644abcb175f1df90a77e50', 'request_id': 'req-131c0905-0d90-4d9d-872b-135c88bad9a7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.173592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] The token cache is being invalidate because project 668309102e644abcb175f1df90a77e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:51.174163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.175064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-131c0905-0d90-4d9d-872b-135c88bad9a7 tempest-VolumeQuotaClassesTest-2064440652 tempest-VolumeQuotaClassesTest-2064440652-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:51.191525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 905/1821] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:50 2022] DELETE /identity/v3/projects/668309102e644abcb175f1df90a77e50 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:51.513034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee335-2243-49fa-9bfe-36dc47387462 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.514121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee335-2243-49fa-9bfe-36dc47387462 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.514554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ee335-2243-49fa-9bfe-36dc47387462 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.516837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 917/1822] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:05:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:51.524201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.524485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.524760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.600816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:51.612529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:51.756554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee335-2243-49fa-9bfe-36dc47387462 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.758160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ec3ee335-2243-49fa-9bfe-36dc47387462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:51.776882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:51.792670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:51.836281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 906/1823] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:51.868872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:51.871204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:51.897030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e271ca-471f-44fb-9119-8366b392e2dd None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.904942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd26580f-b6e4-4d05-826a-00d3b14e767c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.906523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26580f-b6e4-4d05-826a-00d3b14e767c service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.906921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26580f-b6e4-4d05-826a-00d3b14e767c service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.908165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd26580f-b6e4-4d05-826a-00d3b14e767c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.908929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ee335-2243-49fa-9bfe-36dc47387462 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:51.921027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26580f-b6e4-4d05-826a-00d3b14e767c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.924298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd26580f-b6e4-4d05-826a-00d3b14e767c service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:51.927687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:51.928645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:51.929210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:51.929717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] PATH_INFO: `/v3/users/a9fb4ad3ce1e4250a01fc16af3796153` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:51.931435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9fb4ad3ce1e4250a01fc16af3796153)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:51.941159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 907/1824] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:51.955935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:52.074446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9fb4ad3ce1e4250a01fc16af3796153'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.090447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9fb4ad3ce1e4250a01fc16af3796153'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.105702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9fb4ad3ce1e4250a01fc16af3796153'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.129963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9fb4ad3ce1e4250a01fc16af3796153'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.131366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9fb4ad3ce1e4250a01fc16af3796153'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.144168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 918/1825] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:51 2022] DELETE /identity/v3/users/a9fb4ad3ce1e4250a01fc16af3796153 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:52.145937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:52.146360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:52.146638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.201362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:52.204741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:52.439480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 908/1826] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:52.471624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:52.473279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:52.603312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:52.603987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:52.632530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e937b5-ba52-461e-8780-6b725aa53e22 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:52.636764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a507e-268c-427c-a6c9-ff395480357b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:52.642693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:52.643489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:52.643715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:52.644074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] PATH_INFO: `/v3/projects/71692d19f9cb4f11935e9f9c6f81bba5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.646443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77f20e-0af3-44d2-a0be-529d090c3e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:52.647779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71692d19f9cb4f11935e9f9c6f81bba5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:52.648189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77f20e-0af3-44d2-a0be-529d090c3e97 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:52.648513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77f20e-0af3-44d2-a0be-529d090c3e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:52.648720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77f20e-0af3-44d2-a0be-529d090c3e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.652893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:52.669109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77f20e-0af3-44d2-a0be-529d090c3e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:52.671900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77f20e-0af3-44d2-a0be-529d090c3e97 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:52.696571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 919/1827] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:05:52.767301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c909ad9-7300-4b8a-8c27-4259ca961975 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:52.767725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c909ad9-7300-4b8a-8c27-4259ca961975 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:52.768108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c909ad9-7300-4b8a-8c27-4259ca961975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.807631 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71692d19f9cb4f11935e9f9c6f81bba5', 'request_id': 'req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.841088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71692d19f9cb4f11935e9f9c6f81bba5', 'request_id': 'req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.860841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5c909ad9-7300-4b8a-8c27-4259ca961975 None None] MFA Rules not processed for user `714872990c384663bcd564cd5b9ae432`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:52.867731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5c909ad9-7300-4b8a-8c27-4259ca961975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:52.876416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71692d19f9cb4f11935e9f9c6f81bba5', 'request_id': 'req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.920288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] The token cache is being invalidate because project 71692d19f9cb4f11935e9f9c6f81bba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:52.920594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.926967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-132b241f-720e-4ca4-bab7-c9b6d0c41ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:52.932925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 909/1828] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:52 2022] DELETE /identity/v3/projects/71692d19f9cb4f11935e9f9c6f81bba5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:52.938651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:52.940691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:52.940952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:52.982484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:53.001427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:53.002690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:53.047951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 920/1829] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:53.104437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:53.148051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 910/1830] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:53.153637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.153953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.154237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.189742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.192605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:53.203946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:53.205211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:53.222366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 921/1831] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:05:53.224539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f8091c-f287-4e94-aa65-32273d610184 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.226238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.241527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dad31-d93b-4635-b4d0-d0d62f3fc0a3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.251172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.252650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.253397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.253613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.254811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:53.255355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.258621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce5ae9e-e97c-4ad6-936a-c5f3e402975a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.273667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.275721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.277338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.277950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] PATH_INFO: `/v3/projects/5d43deb553414691a425ef3c79b7843b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.281886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 922/1832] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:05:53.284055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d43deb553414691a425ef3c79b7843b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.284263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29b75e-063a-45e5-abf2-5da0af24bda9 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.284841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29b75e-063a-45e5-abf2-5da0af24bda9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.285330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29b75e-063a-45e5-abf2-5da0af24bda9 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.289971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.308375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 923/1833] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:05:53 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:05:53.313218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29b75e-063a-45e5-abf2-5da0af24bda9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.358917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29b75e-063a-45e5-abf2-5da0af24bda9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.367854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3885988-6c8a-4d0d-80aa-75778e8a233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.368812 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3885988-6c8a-4d0d-80aa-75778e8a233f admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.369325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3885988-6c8a-4d0d-80aa-75778e8a233f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.369755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3885988-6c8a-4d0d-80aa-75778e8a233f admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.370602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3885988-6c8a-4d0d-80aa-75778e8a233f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.372768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3885988-6c8a-4d0d-80aa-75778e8a233f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.382467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d43deb553414691a425ef3c79b7843b', 'request_id': 'req-8f14f574-fd35-4d94-8645-210b420cd1b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.398638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d43deb553414691a425ef3c79b7843b', 'request_id': 'req-8f14f574-fd35-4d94-8645-210b420cd1b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.420309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d43deb553414691a425ef3c79b7843b', 'request_id': 'req-8f14f574-fd35-4d94-8645-210b420cd1b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.424372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 924/1834] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:53 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:53.425991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.426275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.426548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.447942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] The token cache is being invalidate because project 5d43deb553414691a425ef3c79b7843b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:53.447942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.448414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.450480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 911/1835] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:53 2022] DELETE /identity/v3/projects/5d43deb553414691a425ef3c79b7843b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:53.451991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.456389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:53.470543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:53.473001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:53.590016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14f574-fd35-4d94-8645-210b420cd1b8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.627334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 925/1836] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:53.628060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.630117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:53.688257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.688891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.689175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.689450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.690170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.692352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.732983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4300ec9d-9d13-4b35-86c1-85e9b608b413 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.739279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.739962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 912/1837] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:53 2022] POST /identity/v3/users => generated 292 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:53.743318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.743610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.743901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] PATH_INFO: `/v3/users/2b480d8de0e34882905792616751c6f6` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.748720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b480d8de0e34882905792616751c6f6)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.751527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.778282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.806013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae3528-3b01-4fe5-bb4a-a179a8e02669 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.813849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.814696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.815038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.815579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.816372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.818253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.832208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:53.856303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 913/1838] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:53.857751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.864314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b480d8de0e34882905792616751c6f6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.876086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b480d8de0e34882905792616751c6f6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.887742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fed299-d141-41eb-b318-6472bc0c8083 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:53.898258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b480d8de0e34882905792616751c6f6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.908122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:53.909776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:53.910370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:53.911658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:53.915230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:53.919520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:53.927175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:53.935670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b480d8de0e34882905792616751c6f6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.938856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b480d8de0e34882905792616751c6f6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:53.956964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 926/1839] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:53 2022] DELETE /identity/v3/users/2b480d8de0e34882905792616751c6f6 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:53.972571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 914/1840] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:53.973482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:53.976844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:54.069751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:54.071789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:54.103616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8cbf6-05eb-4635-85f8-1052f37c6370 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:54.108958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53d068-cc58-4300-93d7-38042548c9d7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:54.123389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed581046-fd76-4c8f-ba7e-3eb10b05b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:54.125340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581046-fd76-4c8f-ba7e-3eb10b05b0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:54.125757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581046-fd76-4c8f-ba7e-3eb10b05b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:54.126783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed581046-fd76-4c8f-ba7e-3eb10b05b0e3 admin admin] PATH_INFO: `/v3/projects/b1ae93f297b84fa0bf3f821e48a7d894/users/1d639576dc9143b5b578bb68ad8451df/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:54.187194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed581046-fd76-4c8f-ba7e-3eb10b05b0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ae93f297b84fa0bf3f821e48a7d894, user_id=1d639576dc9143b5b578bb68ad8451df, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:54.189968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed581046-fd76-4c8f-ba7e-3eb10b05b0e3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:54.257995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 927/1841] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:53 2022] PUT /identity/v3/projects/b1ae93f297b84fa0bf3f821e48a7d894/users/1d639576dc9143b5b578bb68ad8451df/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:54.262719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:54.263033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:54.266310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:54.343729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] MFA Rules not processed for user `1d639576dc9143b5b578bb68ad8451df`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:54.347950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:54.349993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:54.350679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:54.351746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:54.353407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:54.357193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:54.392381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:54.394454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:54.427458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 915/1842] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:54.543689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:54.558614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 928/1843] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:54.570294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b172f-27f7-430e-89b1-2f64a5145947 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:54.734089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:54.735806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:54.736304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:54.736593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:54.738244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:54.741790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:54.818659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 916/1844] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:54 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:54.823909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:54.839083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f6ed8-b88d-482f-b8fc-733b8f3a8bde None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:54.868099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:54.950676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:54.951887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:54.954845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:54.955520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:54.957113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:54.959497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:55.012076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:55.012786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 929/1845] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:54 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:55.031646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc761d0-7af5-42fa-adb7-eb41f01205ca admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:55.048962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:55.052918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:55.053375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:55.053819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:55.054606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:55.059189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:55.104330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:55.116813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 917/1846] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:55.124399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e1c5d-e641-473a-ae85-358edfeecb55 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:55.149861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:55.151654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:55.154433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:55.157821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] PATH_INFO: `/v3/projects/488765db811a4d549ae41f737d939217/users/1ec6a795f5b841998a814b01bfb169bf/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:55.211029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488765db811a4d549ae41f737d939217, user_id=1ec6a795f5b841998a814b01bfb169bf, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:55.218569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:55.262015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 930/1847] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:55 2022] PUT /identity/v3/projects/488765db811a4d549ae41f737d939217/users/1ec6a795f5b841998a814b01bfb169bf/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:55.266656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:55.283689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b8579-9db0-4ef0-8509-d02375a48016 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:55.429267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:55.447225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a608ef-7def-4756-9c93-7461c3189e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:55.448226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a608ef-7def-4756-9c93-7461c3189e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:55.448480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a608ef-7def-4756-9c93-7461c3189e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:55.448866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a608ef-7def-4756-9c93-7461c3189e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:55.450924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a608ef-7def-4756-9c93-7461c3189e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:55.461881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a608ef-7def-4756-9c93-7461c3189e43 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:55.496061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 918/1848] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:55.499985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:55.534127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5d9c5-3259-4df8-9097-cac270082374 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:55.564706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:55.568334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:55.569687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:55.572062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] PATH_INFO: `/v3/projects/488765db811a4d549ae41f737d939217/users/1ec6a795f5b841998a814b01bfb169bf/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:55.581444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=488765db811a4d549ae41f737d939217, user_id=1ec6a795f5b841998a814b01bfb169bf, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:55.589054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:55.647754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:55.648214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:55.648609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:55.652164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 931/1849] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:55 2022] PUT /identity/v3/projects/488765db811a4d549ae41f737d939217/users/1ec6a795f5b841998a814b01bfb169bf/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:55.710540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] MFA Rules not processed for user `1ec6a795f5b841998a814b01bfb169bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:55.712192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:55.924453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 919/1850] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:55.937584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:55.940817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:56.267806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de86621-90aa-4932-8ecb-2d54fd426f70 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:56.298869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:56.299884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:56.300647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:56.301093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:56.339716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:56.343827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:56.416135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 932/1851] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:56.739292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:56.770051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0f47b-8d85-400b-a775-982da6395b94 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:56.817991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:56.819005 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:56.819385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:56.819811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:57.137196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:57.147784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:57.182004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 920/1852] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:57.540000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:57.543387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:57.970137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb6866-f3b5-43f1-af82-95c584fd1da6 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:57.987629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:57.988222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:57.989002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:57.990770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] PATH_INFO: `/v3/projects/35a8738ab6bb4fe19292464f79464dd6` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:57.994637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a8738ab6bb4fe19292464f79464dd6)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:58.000229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:58.087914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:58.415333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a8738ab6bb4fe19292464f79464dd6', 'request_id': 'req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.498290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a8738ab6bb4fe19292464f79464dd6', 'request_id': 'req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.526147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:58.529317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:58.541612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a8738ab6bb4fe19292464f79464dd6', 'request_id': 'req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.585965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] The token cache is being invalidate because project 35a8738ab6bb4fe19292464f79464dd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:05:58.587172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.593301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-218b022f-fe00-4a4d-90dd-6ee86bf9f02c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.601289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 933/1853] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:05:57 2022] DELETE /identity/v3/projects/35a8738ab6bb4fe19292464f79464dd6 => generated 0 bytes in 1062 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:58.603014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e2453e-da70-4990-b046-dfa8da00dfd9 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:58.608937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:58.609729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:58.612485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:58.712399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:58.713867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:58.818822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:58.819990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:58.822556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:58.823029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] PATH_INFO: `/v3/users/abb5577a789042eb9510e4c8567b4dea` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:58.825470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb5577a789042eb9510e4c8567b4dea)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:58.832605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:58.883307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:58.896324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 934/1854] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:58.896324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:58.896324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:58.925751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de855f76-1ebd-454f-af2a-e0a6e2b920c9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:58.951355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:58.952599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:58.953064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:58.953542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] PATH_INFO: `/v3/users/d26124332cbf4577ae65bb50a7ea5b45` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:58.957342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d26124332cbf4577ae65bb50a7ea5b45)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:58.969946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:58.973445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb5577a789042eb9510e4c8567b4dea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.975726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb5577a789042eb9510e4c8567b4dea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:58.997693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb5577a789042eb9510e4c8567b4dea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.052557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb5577a789042eb9510e4c8567b4dea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.090951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd26124332cbf4577ae65bb50a7ea5b45'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.105501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd26124332cbf4577ae65bb50a7ea5b45'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.106106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3e788024-ca08-4d1a-8baf-4b6d33728290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb5577a789042eb9510e4c8567b4dea'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.119598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd26124332cbf4577ae65bb50a7ea5b45'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.125140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 921/1855] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:58 2022] DELETE /identity/v3/users/abb5577a789042eb9510e4c8567b4dea => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:59.128604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.128899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.130291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.144388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd26124332cbf4577ae65bb50a7ea5b45'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.145441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3d6ade0c-ca00-4800-bc4b-d35cea1449d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd26124332cbf4577ae65bb50a7ea5b45'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:05:59.154566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 935/1856] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:05:58 2022] DELETE /identity/v3/users/d26124332cbf4577ae65bb50a7ea5b45 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:05:59.157377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2826896-d1e7-4922-bdbb-0a1e9a4d84ad None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.157792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2826896-d1e7-4922-bdbb-0a1e9a4d84ad None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.158182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2826896-d1e7-4922-bdbb-0a1e9a4d84ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.187977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e2826896-d1e7-4922-bdbb-0a1e9a4d84ad None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:59.189933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e2826896-d1e7-4922-bdbb-0a1e9a4d84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:59.198122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:59.203037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:59.207636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:59.306501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 936/1857] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:59.306501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.306501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.306501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.330604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:05:59.332230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:59.341914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 937/1858] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:59.372089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 922/1859] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:05:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:05:59.378603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.395335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed16df7-bf77-4a1f-bd57-2a4b955389f1 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.403819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.406341 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.407072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.407711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.408681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:05:59.409453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.412227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.433785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 938/1860] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:05:59.443533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.455743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c75f5-9b62-4826-999d-cee3382e609b None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.476665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.476665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.476665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.476665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.477391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.484489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.561957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 923/1861] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:05:59 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:05:59.569641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.582253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.587526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:05:59.602546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef9b76-963e-4ba2-a06c-4a6e8e2fa06a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.723747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a955e4-c9a4-4a8f-8268-d8135e8908e5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.729483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.731283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.732933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.734191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.735031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.735825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.736136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.736501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.738295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.742926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.761708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.766095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.781819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 939/1862] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:05:59 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:05:59.781917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.788539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 924/1863] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:59.790953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.793960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b5e2e-d399-4b4f-9ba0-46aa4aa1a04f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.810080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.811978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.812697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.814381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.826551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476295b0-b652-49cb-9c03-7dcb94b454ab service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.829827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.831336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.843300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 940/1864] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:05:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:05:59.845214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.846290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.846575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.846897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.847530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.849333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.862639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 925/1865] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:59.870785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.884059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56758d39-df8c-404b-b787-d0f4727da211 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.892162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe43c73-cfb7-4518-82cb-429bf8572fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.892950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe43c73-cfb7-4518-82cb-429bf8572fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.893309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe43c73-cfb7-4518-82cb-429bf8572fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.894424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe43c73-cfb7-4518-82cb-429bf8572fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.895465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe43c73-cfb7-4518-82cb-429bf8572fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.897906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe43c73-cfb7-4518-82cb-429bf8572fe5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:05:59.916130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 941/1866] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:05:59.921715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:05:59.934305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6974d-c82a-4105-8439-7c7ad1c756b6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:05:59.941203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:05:59.941915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:05:59.942250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:05:59.942595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] PATH_INFO: `/v3/projects/6e3c03ab94c741608e0d26f6d6161a97/users/8a9c89b3718c4116bd045923e335b0a1/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:05:59.955430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:05:59.993647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e3c03ab94c741608e0d26f6d6161a97, user_id=8a9c89b3718c4116bd045923e335b0a1, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:05:59.997599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:00.036906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 926/1867] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:05:59 2022] PUT /identity/v3/projects/6e3c03ab94c741608e0d26f6d6161a97/users/8a9c89b3718c4116bd045923e335b0a1/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:00.041651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:00.041917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:00.042291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:00.116133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] MFA Rules not processed for user `8a9c89b3718c4116bd045923e335b0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:00.122615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:00.268611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:00.296906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 942/1868] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:00.299164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:00.418281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86da12-5629-496d-b34d-d40c734514d7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:00.425646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:00.426810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:00.427195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:00.427572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:00.428561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:00.430659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:00.471264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 927/1869] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:00 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:00.475121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:00.507610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff6718-3c29-4921-8b8d-df9c00c53e37 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:00.525044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:00.527624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:00.633763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:00.635261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:00.635649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:00.636176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:00.637227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:00.642206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:00.643424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1813cf6-7cd7-4ea2-9edd-17b655e93fc7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:00.648449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:00.649354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:00.652052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:00.652052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] PATH_INFO: `/v3/projects/8e609628bade4e388ae76a865dddf570` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:00.652052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e609628bade4e388ae76a865dddf570)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:00.653483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:00.692213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 943/1870] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:00 2022] POST /identity/v3/users => generated 289 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:00.694306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:00.698611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:00.720459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e609628bade4e388ae76a865dddf570', 'request_id': 'req-7e1b9dfa-531e-4f98-9bea-50a50fa31910'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.724631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fead4c7a-8143-454d-8a2e-5b022b0c04be admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:00.737236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e609628bade4e388ae76a865dddf570', 'request_id': 'req-7e1b9dfa-531e-4f98-9bea-50a50fa31910'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.747388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e609628bade4e388ae76a865dddf570', 'request_id': 'req-7e1b9dfa-531e-4f98-9bea-50a50fa31910'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.766478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:00.792400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] The token cache is being invalidate because project 8e609628bade4e388ae76a865dddf570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:00.792400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.792400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.793866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 928/1871] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:00 2022] DELETE /identity/v3/projects/8e609628bade4e388ae76a865dddf570 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:00.803606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:00.803606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:00.828075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:00.828951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:00.829310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:00.829660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] PATH_INFO: `/v3/projects/cd08c3763d1f4a1c8f045d0cee00f604` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:00.831181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd08c3763d1f4a1c8f045d0cee00f604)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:00.833668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:00.908794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b9dfa-531e-4f98-9bea-50a50fa31910 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:00.916692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd08c3763d1f4a1c8f045d0cee00f604', 'request_id': 'req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.933243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd08c3763d1f4a1c8f045d0cee00f604', 'request_id': 'req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.947472 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd08c3763d1f4a1c8f045d0cee00f604', 'request_id': 'req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.971390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] The token cache is being invalidate because project cd08c3763d1f4a1c8f045d0cee00f604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:00.971867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.972933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:00.975622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 944/1872] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:00 2022] DELETE /identity/v3/projects/cd08c3763d1f4a1c8f045d0cee00f604 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:00.977022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:00.982540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:00.997661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d708c-4dd5-441c-bda3-9e80548511d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:00.998428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d708c-4dd5-441c-bda3-9e80548511d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.000153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d708c-4dd5-441c-bda3-9e80548511d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.001308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d708c-4dd5-441c-bda3-9e80548511d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.004997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:01.005957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d708c-4dd5-441c-bda3-9e80548511d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.009978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d708c-4dd5-441c-bda3-9e80548511d9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.040148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 929/1873] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:01.041930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.042503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.042964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.085613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:01.088744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.098353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab34e5-6b48-4c6b-9a5b-785ea19e9dca admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.113119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 930/1874] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:01.113119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.113119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.116816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd422b-8de4-47ea-b289-2043e52809a9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.117261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.118330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.118681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.119031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] PATH_INFO: `/v3/users/b920e99eecb546b7a08047f853220423` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.120328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b920e99eecb546b7a08047f853220423)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.122063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.122481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.122903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.123291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.123761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] PATH_INFO: `/v3/projects/b5ca51cc61094865a08aa74d316802a8/users/d382d7ac4c2f49bba4e96d16381a5073/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.148499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ca51cc61094865a08aa74d316802a8, user_id=d382d7ac4c2f49bba4e96d16381a5073, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.153794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.175190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 931/1875] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:01 2022] PUT /identity/v3/projects/b5ca51cc61094865a08aa74d316802a8/users/d382d7ac4c2f49bba4e96d16381a5073/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:01.179950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.179950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.185323 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b920e99eecb546b7a08047f853220423'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.193466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b920e99eecb546b7a08047f853220423'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.203135 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b920e99eecb546b7a08047f853220423'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.218345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b920e99eecb546b7a08047f853220423'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.219106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-eaf23455-3a69-4fe7-909f-d4bc71697d12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b920e99eecb546b7a08047f853220423'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.225764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 945/1876] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:00 2022] DELETE /identity/v3/users/b920e99eecb546b7a08047f853220423 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:01.227036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.227324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.227603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.250203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:01.251623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.276411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4f983-5cda-4e26-a2ec-b51393299c02 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.292442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:01.352901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.353835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.353969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.354097 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 946/1877] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:01.354210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] PATH_INFO: `/v3/users/1d639576dc9143b5b578bb68ad8451df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.355643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d639576dc9143b5b578bb68ad8451df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.357709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.360853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.363078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.384723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b6a3e-f66f-4e2b-a99b-20becd5e378f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.395806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.396171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.396932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.397475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.398865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.402134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.426384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d639576dc9143b5b578bb68ad8451df'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.427110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d639576dc9143b5b578bb68ad8451df'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.428760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 947/1878] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:01 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:01.430420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.437586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d639576dc9143b5b578bb68ad8451df'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.441822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12239074-7aee-481e-be25-b3ee9edc6098 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.447321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.447867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.449335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.449335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.449335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:01.449335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.450961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.455279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d639576dc9143b5b578bb68ad8451df'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.476133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 948/1879] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:01.476133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.476484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d639576dc9143b5b578bb68ad8451df'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.488862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab74dd8-7167-481f-8dd0-f9e5a8f5c6e1 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.489214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 932/1880] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:01 2022] DELETE /identity/v3/users/1d639576dc9143b5b578bb68ad8451df => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:01.490901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.493361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.499924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:01.582149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464fdb0-2c33-4d2c-85dc-c93518093c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.582998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464fdb0-2c33-4d2c-85dc-c93518093c10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.583341 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464fdb0-2c33-4d2c-85dc-c93518093c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.583556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fda2e-8e8a-44c8-ab6e-08dfdf33403b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.583759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464fdb0-2c33-4d2c-85dc-c93518093c10 admin admin] PATH_INFO: `/v3/projects/b5ca51cc61094865a08aa74d316802a8/users/d382d7ac4c2f49bba4e96d16381a5073/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.586628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464fdb0-2c33-4d2c-85dc-c93518093c10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ca51cc61094865a08aa74d316802a8, user_id=d382d7ac4c2f49bba4e96d16381a5073, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.589601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464fdb0-2c33-4d2c-85dc-c93518093c10 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.593825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.595461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.595461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.595965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.597251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.600966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.601558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 949/1881] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:01 2022] PUT /identity/v3/projects/b5ca51cc61094865a08aa74d316802a8/users/d382d7ac4c2f49bba4e96d16381a5073/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:01.603263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1a1c-9068-4ee8-bb4c-0dc52305e259 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.603263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1a1c-9068-4ee8-bb4c-0dc52305e259 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.603511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e1a1c-9068-4ee8-bb4c-0dc52305e259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.623781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a36e1a1c-9068-4ee8-bb4c-0dc52305e259 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:01.624884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a36e1a1c-9068-4ee8-bb4c-0dc52305e259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.663660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 933/1882] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:01.663660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.664317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.693411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 950/1883] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:01.694795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.695301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.695638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.721555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:01.728978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] MFA Rules not processed for user `d382d7ac4c2f49bba4e96d16381a5073`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:01.730094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.767441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f9308-c460-4e06-bdeb-a6b2d140e004 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.775840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:01.777173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:01.777766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:01.778339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] PATH_INFO: `/v3/projects/75d28216713146cd9a92d7d10b1c19b4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:01.780892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75d28216713146cd9a92d7d10b1c19b4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:01.782863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:01.848725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 951/1884] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:01.849645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.887473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75d28216713146cd9a92d7d10b1c19b4', 'request_id': 'req-9256ac6f-866e-4867-9741-050803faa203'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.888083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feaab8-3d75-47bf-a6bf-efea22b196be None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:01.906859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75d28216713146cd9a92d7d10b1c19b4', 'request_id': 'req-9256ac6f-866e-4867-9741-050803faa203'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.915829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75d28216713146cd9a92d7d10b1c19b4', 'request_id': 'req-9256ac6f-866e-4867-9741-050803faa203'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.940727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] The token cache is being invalidate because project 75d28216713146cd9a92d7d10b1c19b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:01.941031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.941816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:01.946894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 934/1885] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:01 2022] DELETE /identity/v3/projects/75d28216713146cd9a92d7d10b1c19b4 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:01.947912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:01.949508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:01.977587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:02.026062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.026990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.027455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.027997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.028839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.031920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.074439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 952/1886] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:01 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:02.074439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.075970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.101219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256ac6f-866e-4867-9741-050803faa203 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.105267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.105890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.106237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.106547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.108804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.135387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.137567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66edab44-2da5-44b5-aa47-6f1baffb616c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.157106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 935/1887] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:02.167492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.167788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.170684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.179551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce124e5-d4d0-404f-be82-83c68eba8338 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.185767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.187611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.187896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.188280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.190611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.199022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:02.201026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.211772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 936/1888] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:02.212808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.214389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.235654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1991f5-5eb1-4154-a70d-da5f15875a37 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.243920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e733b-6a26-44b7-a67e-aad8b5a2d54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.245270 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e733b-6a26-44b7-a67e-aad8b5a2d54f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.245619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e733b-6a26-44b7-a67e-aad8b5a2d54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.245895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e733b-6a26-44b7-a67e-aad8b5a2d54f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.246572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e733b-6a26-44b7-a67e-aad8b5a2d54f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.248969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e733b-6a26-44b7-a67e-aad8b5a2d54f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.264705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 937/1889] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:02.266287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.266579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.266856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.306461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:02.309126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.327258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 938/1890] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:02.328598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.348283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2cbc86-5575-4d66-a45d-c4040cf559e3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.360575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.362034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.362932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.364255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.365679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.369741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.393072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 939/1891] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:02.393833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.399331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.404040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.413905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675f58c8-9ecd-46cd-ad9b-a14559ea85db admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.419556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.420274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.421218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.421568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.422309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:02.422690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.423277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 953/1892] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:02.424489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.424761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.443003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 940/1893] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:02.447894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.450596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc25014-2758-4d83-8456-250c4d4ce365 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.470614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276224a-b7b5-4453-88b5-06b01865011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.471411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276224a-b7b5-4453-88b5-06b01865011e admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.471937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276224a-b7b5-4453-88b5-06b01865011e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.472172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276224a-b7b5-4453-88b5-06b01865011e admin admin] PATH_INFO: `/v3/projects/e31b23c48bd44baa8a56f4379babeaac/users/45ae30ab96db4d9ca883df3deba00693/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.478195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ef1cd-eb6d-468f-aba7-7df0dbad95ad admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.505754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276224a-b7b5-4453-88b5-06b01865011e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e31b23c48bd44baa8a56f4379babeaac, user_id=45ae30ab96db4d9ca883df3deba00693, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.506698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.507736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.508281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.508753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.509621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276224a-b7b5-4453-88b5-06b01865011e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.515861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:02.522282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:02.534801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:02.568903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 954/1894] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:02 2022] PUT /identity/v3/projects/e31b23c48bd44baa8a56f4379babeaac/users/45ae30ab96db4d9ca883df3deba00693/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:02.575427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.575854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.576489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.635071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 941/1895] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:02 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:02.636261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.636399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] MFA Rules not processed for user `45ae30ab96db4d9ca883df3deba00693`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:02.640274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.722948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:02.765217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a974950-db61-4f95-a981-d78f742056f5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:02.771705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:02.790071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:02.790546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:02.790999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:02.797495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 955/1896] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:02.798304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:02.805498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:02.999744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71b381f-f86f-4474-aa31-28f5eeff1e3c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.000686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.002669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.011189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.012613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.012914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.016408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.019379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.024797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.029277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 942/1897] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:03.031676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.066816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d356d-a74c-43d8-971e-230798cbe78c service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.073260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.074024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.074540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.074797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.075759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.078433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.116886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 956/1898] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:02 2022] POST /identity/v3/users => generated 294 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:03.118099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.122965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:03.149612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 943/1899] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:03 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:03.151289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.158002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba44e87-6a66-46b3-8af4-00322cfcc9da admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.330035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb880bbf-ad7e-4f66-97ef-93583d22ad2a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.342430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.345588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.348314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.348743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.349616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.355017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.388609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.391939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.392329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 944/1900] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:03.394102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.394282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.398145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] PATH_INFO: `/v3/projects/b1ae93f297b84fa0bf3f821e48a7d894` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.401006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ae93f297b84fa0bf3f821e48a7d894)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.414492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.426217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9899e-209f-44b8-bbf5-7644ecc43948 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.443090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.446490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.446490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.446490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.446490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.453105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.492550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:03.522625 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 945/1901] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:03 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:03.523355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.569518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163321d-ebf9-4527-8ebc-ec9b3a8180cd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.605716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ae93f297b84fa0bf3f821e48a7d894', 'request_id': 'req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:03.622852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ae93f297b84fa0bf3f821e48a7d894', 'request_id': 'req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:03.650011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ae93f297b84fa0bf3f821e48a7d894', 'request_id': 'req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:03.653358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:03.693804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] The token cache is being invalidate because project b1ae93f297b84fa0bf3f821e48a7d894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:03.694531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:03.695536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:03.709016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 957/1902] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:03 2022] DELETE /identity/v3/projects/b1ae93f297b84fa0bf3f821e48a7d894 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:03.710172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.717018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:03.875564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.876351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.876904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.877394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.878657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.882807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.903313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 946/1903] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:03.907464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.910187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:03.933707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3786601-d31e-4919-b952-afea5effa02e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.933918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d6e90-c8c9-4d1e-8aa5-8636568e04ae admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:03.942659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.943630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.944201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.944698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] PATH_INFO: `/v3/users/1ec6a795f5b841998a814b01bfb169bf` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.946460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:03.946850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ec6a795f5b841998a814b01bfb169bf)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.947146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:03.947436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:03.948364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:03.950273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:03.950512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.954033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:03.982500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 958/1904] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:03.982500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:03.989110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.035542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7cc31b-d893-4ec0-8378-48955e7296ac admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.050693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:04.052786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.054418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.054418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] PATH_INFO: `/v3/projects/94b86d7d3888416380197ecea25412bf/users/bd7508cea63c44e5b2aaf3ccdbd061d8/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.065719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ec6a795f5b841998a814b01bfb169bf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.068963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ec6a795f5b841998a814b01bfb169bf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.079809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b86d7d3888416380197ecea25412bf, user_id=bd7508cea63c44e5b2aaf3ccdbd061d8, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:04.084758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:04.097445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ec6a795f5b841998a814b01bfb169bf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.109672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 959/1905] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:03 2022] PUT /identity/v3/projects/94b86d7d3888416380197ecea25412bf/users/bd7508cea63c44e5b2aaf3ccdbd061d8/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:04.111319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:04.148130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ec6a795f5b841998a814b01bfb169bf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.175852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8d935cf5-68d6-4d06-b04a-8c86e111cb9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ec6a795f5b841998a814b01bfb169bf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:04.202233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 947/1906] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:03 2022] DELETE /identity/v3/users/1ec6a795f5b841998a814b01bfb169bf => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:04.213089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.213462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.213814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.326209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] MFA Rules not processed for user `bd7508cea63c44e5b2aaf3ccdbd061d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:04.331373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.363649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826340f2-8081-4499-a532-905d2e313c25 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.605062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:04.640777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:04.641381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.641664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.641944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] PATH_INFO: `/v3/projects/2329bd36f61a497aabf6398da1883aa7/users/6a6ff6f85cec42518c38bc7456b24d5c/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.649263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 948/1907] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:04.652397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:04.656536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.694887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2329bd36f61a497aabf6398da1883aa7, user_id=6a6ff6f85cec42518c38bc7456b24d5c, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:04.698732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:04.703873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:04.706780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbe9314-fd56-4ffd-8c08-d623f8d5e0c2 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.717662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:04.718776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.719240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.720420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.721696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:04.726457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:04.735234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 960/1908] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:04 2022] PUT /identity/v3/projects/2329bd36f61a497aabf6398da1883aa7/users/6a6ff6f85cec42518c38bc7456b24d5c/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:04.737440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:04.741576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.821714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 949/1909] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:04 2022] POST /identity/v3/projects => generated 385 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:04.822755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:04.831119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:04.925553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e024e6-ce7c-4601-a8dc-fa2a5cdceee6 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.957788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb1d9f-013f-4c4d-bb70-fdc083bd36cb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:04.964489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:04.965322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:04.965758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:04.966141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:04.966940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:04.969643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:04.990279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 950/1910] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:04.993303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:05.025653 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43465a7a-a438-4ea6-985b-a4227a09f76c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:05.046805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:05.047520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.047728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.048025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.048992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:05.054839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:05.088388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:05.089470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.089828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.090338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] PATH_INFO: `/v3/projects/488765db811a4d549ae41f737d939217` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.092144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] RBAC: Authorizing `identity:delete_project(project_id=488765db811a4d549ae41f737d939217)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:05.095358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:05.108266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 951/1911] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:04 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:05.109493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:05.152939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060b928-8729-48ce-87bc-e366ac2d2a34 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:05.163587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:05.164812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.165276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.165783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] PATH_INFO: `/v3/projects/2329bd36f61a497aabf6398da1883aa7/users/6a6ff6f85cec42518c38bc7456b24d5c/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.173471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2329bd36f61a497aabf6398da1883aa7, user_id=6a6ff6f85cec42518c38bc7456b24d5c, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:05.179805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:05.228108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 952/1912] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:05 2022] PUT /identity/v3/projects/2329bd36f61a497aabf6398da1883aa7/users/6a6ff6f85cec42518c38bc7456b24d5c/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:05.228108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:05.266213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '488765db811a4d549ae41f737d939217', 'request_id': 'req-137e5cfc-ff4e-4590-9f66-cd368132c73a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.301282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '488765db811a4d549ae41f737d939217', 'request_id': 'req-137e5cfc-ff4e-4590-9f66-cd368132c73a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.331309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '488765db811a4d549ae41f737d939217', 'request_id': 'req-137e5cfc-ff4e-4590-9f66-cd368132c73a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.385138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] The token cache is being invalidate because project 488765db811a4d549ae41f737d939217 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:05.387016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.393673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-137e5cfc-ff4e-4590-9f66-cd368132c73a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:05.404510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 961/1913] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:04 2022] DELETE /identity/v3/projects/488765db811a4d549ae41f737d939217 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:05.406260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c90e6-e240-42dc-955b-2c736c91e0d6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.407771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c90e6-e240-42dc-955b-2c736c91e0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.409735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c90e6-e240-42dc-955b-2c736c91e0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.449463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7713a5-fbec-4bec-8260-d8bc0c55c6a7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:05.484360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-bb5c90e6-e240-42dc-955b-2c736c91e0d6 None None] MFA Rules not processed for user `6a6ff6f85cec42518c38bc7456b24d5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:05.485953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bb5c90e6-e240-42dc-955b-2c736c91e0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:05.557846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:05.657831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:05.710579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:05.711627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.715040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.716413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.719205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:05.726053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:05.750330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 962/1914] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:05.753094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.753398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.753673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.767647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 953/1915] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 546 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:05.772560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:05.776686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:05.823775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeec52-7abb-4000-b6cc-a5c657dd378e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:05.834129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:05.835929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:05.846874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355aad9-4184-4238-bc23-8b60f2d7fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:05.852128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 963/1916] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:05.852394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355aad9-4184-4238-bc23-8b60f2d7fa4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.852394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355aad9-4184-4238-bc23-8b60f2d7fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.852667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:05.852797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355aad9-4184-4238-bc23-8b60f2d7fa4e admin admin] PATH_INFO: `/v3/projects/df34f344952740018d6948182a27ef8a/users/4509ab2fe91542cba262d192e31cd485/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.852932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:05.888553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355aad9-4184-4238-bc23-8b60f2d7fa4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34f344952740018d6948182a27ef8a, user_id=4509ab2fe91542cba262d192e31cd485, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:05.895269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355aad9-4184-4238-bc23-8b60f2d7fa4e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:05.930382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 954/1917] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:05 2022] PUT /identity/v3/projects/df34f344952740018d6948182a27ef8a/users/4509ab2fe91542cba262d192e31cd485/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:05.933084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:05.934059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:05.934634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:05.948059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd6733-0652-4704-abd1-86e3cbf024c7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:05.988842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:05.991357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:06.033934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.033934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.033934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.033934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.035689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:06.128347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.129803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.143610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 964/1918] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:06.144910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.183022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:06.221975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 955/1919] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:06.226169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.249963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ecc680-2a97-488e-baee-af68f8ac4b9d None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.250804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261c12e-a6d5-474f-973e-5429c22db9c4 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.260999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.262245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.262653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.263040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.264046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.265689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.266062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.266339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.267219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.268965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.270621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:06.270961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.275421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.282693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 965/1920] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:06.286987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.301665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052646d-4ab5-47c0-a12e-162bd92fbcbf admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.319951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 956/1921] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:06.320561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b99499e-0906-45e1-a7bd-90253bc52118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.320944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.321593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99499e-0906-45e1-a7bd-90253bc52118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.322141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99499e-0906-45e1-a7bd-90253bc52118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.322530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b99499e-0906-45e1-a7bd-90253bc52118 admin admin] PATH_INFO: `/v3/projects/df34f344952740018d6948182a27ef8a/users/4509ab2fe91542cba262d192e31cd485/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.331326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99499e-0906-45e1-a7bd-90253bc52118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34f344952740018d6948182a27ef8a, user_id=4509ab2fe91542cba262d192e31cd485, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.335442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b99499e-0906-45e1-a7bd-90253bc52118 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.346232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f38c-0de3-401a-9521-d1b9e36cb69f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.376992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 966/1922] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:06 2022] PUT /identity/v3/projects/df34f344952740018d6948182a27ef8a/users/4509ab2fe91542cba262d192e31cd485/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:06.378550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.381165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.381352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.463247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] MFA Rules not processed for user `4509ab2fe91542cba262d192e31cd485`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:06.464533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:06.524059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.524690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.525011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.525237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.526301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.532883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.600722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 957/1923] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:06 2022] POST /identity/v3/projects => generated 377 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:06.605030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.648794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 967/1924] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:06.673621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.676142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:06.684201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:06.770265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d62c13-b87b-4915-9490-dbdd006b2c54 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.782960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.782960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.782960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.782960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.783814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.785746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.817022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1adb0-8667-4168-93dd-3d2364ba9822 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.847437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 958/1925] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:06 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:06.849107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.880332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac023160-40ee-4eb9-8611-d2a70ec350d9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.892604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.893621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.894463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.894930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.895774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.898419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:06.921475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 959/1926] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:06.926188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:06.953816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:06.961655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5a843-7419-4ad8-bb1b-074bd615f8d6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:06.969569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.970906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.971596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.972532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.978914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:06.979934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:06.980306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:06.980587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:06.981266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:06.985655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:07.034126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 960/1927] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:07.035754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:07.081703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d551cfe-bad2-4a02-9ae8-8cf678d662f0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:07.100184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989f418-79fc-4c37-a071-031eb18f2f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:07.101611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f418-79fc-4c37-a071-031eb18f2f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:07.102128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f418-79fc-4c37-a071-031eb18f2f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:07.102769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989f418-79fc-4c37-a071-031eb18f2f59 admin admin] PATH_INFO: `/v3/projects/7fc65b04013f453c99dde9d2ae73889e/users/46f92714bf894f3ab5d60a2240dd5068/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:07.138113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989f418-79fc-4c37-a071-031eb18f2f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fc65b04013f453c99dde9d2ae73889e, user_id=46f92714bf894f3ab5d60a2240dd5068, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:07.146608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1989f418-79fc-4c37-a071-031eb18f2f59 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:07.207937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 961/1928] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:07 2022] PUT /identity/v3/projects/7fc65b04013f453c99dde9d2ae73889e/users/46f92714bf894f3ab5d60a2240dd5068/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:07.234354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6c82b-acdf-488a-861f-1ff3c7dd47da None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:07.235478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6c82b-acdf-488a-861f-1ff3c7dd47da None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:07.236892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6c82b-acdf-488a-861f-1ff3c7dd47da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:07.262356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:07.266350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:07.300836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 968/1929] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:07.385863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-96d6c82b-acdf-488a-861f-1ff3c7dd47da None None] MFA Rules not processed for user `46f92714bf894f3ab5d60a2240dd5068`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:07.387992 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-96d6c82b-acdf-488a-861f-1ff3c7dd47da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:07.652952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 962/1930] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:07.677769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:07.697263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab49ca-531e-4a24-a102-33a829d127e8 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:07.804809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c042e9-7ad2-428e-9e77-c50e54da26cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:07.806146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c042e9-7ad2-428e-9e77-c50e54da26cc service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:07.806687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c042e9-7ad2-428e-9e77-c50e54da26cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:07.807436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c042e9-7ad2-428e-9e77-c50e54da26cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:07.821818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c042e9-7ad2-428e-9e77-c50e54da26cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:07.827781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c042e9-7ad2-428e-9e77-c50e54da26cc service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:07.841231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 969/1931] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:08.556977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca513fc-b6e6-47b4-bd83-5611f42b2a14 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:08.558073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca513fc-b6e6-47b4-bd83-5611f42b2a14 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:08.559376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca513fc-b6e6-47b4-bd83-5611f42b2a14 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:08.561618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 963/1932] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:06:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:06:08.706607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:08.707491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:08.707491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:08.823880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:08.831094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:09.158400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 970/1933] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:06:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:09.304607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca513fc-b6e6-47b4-bd83-5611f42b2a14 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:09.403709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:09.406132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:09.434240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781eac18-a4e2-4244-8002-e3c6b2104e18 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:09.456889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad182002-43e3-473a-881f-9e198b3fd648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:09.459507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad182002-43e3-473a-881f-9e198b3fd648 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:09.460587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad182002-43e3-473a-881f-9e198b3fd648 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:09.463047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad182002-43e3-473a-881f-9e198b3fd648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:09.491752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad182002-43e3-473a-881f-9e198b3fd648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:09.494127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad182002-43e3-473a-881f-9e198b3fd648 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:09.518682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 971/1934] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:09.604835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca513fc-b6e6-47b4-bd83-5611f42b2a14 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:09.631779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:09.633586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:09.635547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:09.636455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] PATH_INFO: `/v3/users/bd7508cea63c44e5b2aaf3ccdbd061d8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:09.642439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7508cea63c44e5b2aaf3ccdbd061d8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:09.652385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:09.678982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:09.810657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7508cea63c44e5b2aaf3ccdbd061d8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:09.811889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7508cea63c44e5b2aaf3ccdbd061d8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:09.835180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7508cea63c44e5b2aaf3ccdbd061d8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:09.868097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7508cea63c44e5b2aaf3ccdbd061d8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:09.883637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7508cea63c44e5b2aaf3ccdbd061d8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:09.899366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 964/1935] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:09 2022] DELETE /identity/v3/users/bd7508cea63c44e5b2aaf3ccdbd061d8 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:09.905123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:09.905858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:09.906193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:09.977282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:10.002177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:10.027261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:10.094001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:10.095809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:10.156730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 972/1936] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:10.166295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:10.168894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:10.233925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08c02e-5cc3-485d-9c6a-c760ad1c3ac3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:10.244518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:10.246853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:10.246943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:10.259369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2077262-fcaf-4cf1-a174-cce08968f2d7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:10.259984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] PATH_INFO: `/v3/users/45ae30ab96db4d9ca883df3deba00693` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:10.264254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ae30ab96db4d9ca883df3deba00693)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:10.266021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4d78f-0f6b-4346-bad9-d73be9e62d78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:10.268166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:10.269111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4d78f-0f6b-4346-bad9-d73be9e62d78 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:10.269618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4d78f-0f6b-4346-bad9-d73be9e62d78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:10.270241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4d78f-0f6b-4346-bad9-d73be9e62d78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:10.292261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4d78f-0f6b-4346-bad9-d73be9e62d78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:10.294681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4d78f-0f6b-4346-bad9-d73be9e62d78 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:10.348185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 973/1937] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:10.445294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ae30ab96db4d9ca883df3deba00693'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.449029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ae30ab96db4d9ca883df3deba00693'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.484268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ae30ab96db4d9ca883df3deba00693'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.515979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ae30ab96db4d9ca883df3deba00693'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.532079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ae30ab96db4d9ca883df3deba00693'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.544346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:10.553162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 965/1938] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:10 2022] DELETE /identity/v3/users/45ae30ab96db4d9ca883df3deba00693 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:10.557324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:10.557644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:10.557875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:10.579316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:10.581895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:10.689715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 974/1939] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:10.709140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:10.710825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:10.803873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c40e3-d9e2-4d41-ad0c-8977c7df83bc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:10.807522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:10.808182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:10.808465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:10.808746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:10.819451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:10.822675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:10.834286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 966/1940] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:10.838929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:10.840482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:10.854045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21cd9b6-9a93-4ee6-a378-79b7f1eedb28 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:10.860270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:10.860924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:10.861743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:10.862084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] PATH_INFO: `/v3/projects/94b86d7d3888416380197ecea25412bf` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:10.863310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b86d7d3888416380197ecea25412bf)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:10.865487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:10.931074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b86d7d3888416380197ecea25412bf', 'request_id': 'req-3f11b100-3647-4a09-a3de-24b981d72f39'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.941393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b86d7d3888416380197ecea25412bf', 'request_id': 'req-3f11b100-3647-4a09-a3de-24b981d72f39'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.944542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:10.949454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b86d7d3888416380197ecea25412bf', 'request_id': 'req-3f11b100-3647-4a09-a3de-24b981d72f39'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.967859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] The token cache is being invalidate because project 94b86d7d3888416380197ecea25412bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:10.967859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.968855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:10.970792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 975/1941] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:10 2022] DELETE /identity/v3/projects/94b86d7d3888416380197ecea25412bf => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:10.974940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:10.987329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17fa01-668a-4bfd-b524-544ec8212af9 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:11.099454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:11.100209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:11.100532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:11.101009 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] PATH_INFO: `/v3/users/4509ab2fe91542cba262d192e31cd485` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:11.103173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4509ab2fe91542cba262d192e31cd485)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:11.106176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:11.175006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4509ab2fe91542cba262d192e31cd485'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.175848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4509ab2fe91542cba262d192e31cd485'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.185340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4509ab2fe91542cba262d192e31cd485'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.225128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4509ab2fe91542cba262d192e31cd485'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.237242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4509ab2fe91542cba262d192e31cd485'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.251616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 967/1942] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:10 2022] DELETE /identity/v3/users/4509ab2fe91542cba262d192e31cd485 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:11.259935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:11.261527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:11.289120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11b100-3647-4a09-a3de-24b981d72f39 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:11.379312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:11.379844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:11.380159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:11.380438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] PATH_INFO: `/v3/projects/e31b23c48bd44baa8a56f4379babeaac` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:11.381723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e31b23c48bd44baa8a56f4379babeaac)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:11.383744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:11.476716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e31b23c48bd44baa8a56f4379babeaac', 'request_id': 'req-ccfb6d8e-c309-4448-ad20-6e84464812bb'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.489383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e31b23c48bd44baa8a56f4379babeaac', 'request_id': 'req-ccfb6d8e-c309-4448-ad20-6e84464812bb'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.503275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e31b23c48bd44baa8a56f4379babeaac', 'request_id': 'req-ccfb6d8e-c309-4448-ad20-6e84464812bb'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.528753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] The token cache is being invalidate because project e31b23c48bd44baa8a56f4379babeaac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:11.529276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.530626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.533160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 976/1943] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:11 2022] DELETE /identity/v3/projects/e31b23c48bd44baa8a56f4379babeaac => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:11.539690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:11.542259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:11.570627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b2781-aba7-47f0-bb1d-fc6c473d3445 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:11.624346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:11.693206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:11.694405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:11.694898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:11.695380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] PATH_INFO: `/v3/users/6a6ff6f85cec42518c38bc7456b24d5c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:11.697670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6ff6f85cec42518c38bc7456b24d5c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:11.701790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:11.806473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6ff6f85cec42518c38bc7456b24d5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.807557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6ff6f85cec42518c38bc7456b24d5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.823353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6ff6f85cec42518c38bc7456b24d5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.855173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6ff6f85cec42518c38bc7456b24d5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.872784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6ff6f85cec42518c38bc7456b24d5c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:11.884379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 968/1944] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:11 2022] DELETE /identity/v3/users/6a6ff6f85cec42518c38bc7456b24d5c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:12.055541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.058340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:12.109551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb6d8e-c309-4448-ad20-6e84464812bb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:12.244962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:12.245633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.245920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.246199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] PATH_INFO: `/v3/projects/df34f344952740018d6948182a27ef8a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.247563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df34f344952740018d6948182a27ef8a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:12.249932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:12.274696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:12.358447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df34f344952740018d6948182a27ef8a', 'request_id': 'req-0565d0c6-1c55-4db5-8d37-31f660c98fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.370949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df34f344952740018d6948182a27ef8a', 'request_id': 'req-0565d0c6-1c55-4db5-8d37-31f660c98fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.373122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.374714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:12.380978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df34f344952740018d6948182a27ef8a', 'request_id': 'req-0565d0c6-1c55-4db5-8d37-31f660c98fb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.392692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a43b41-5cd8-4c93-922d-626fc1cda7c7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:12.408474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] The token cache is being invalidate because project df34f344952740018d6948182a27ef8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:12.410196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.411343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0565d0c6-1c55-4db5-8d37-31f660c98fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.414657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 977/1945] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:12 2022] DELETE /identity/v3/projects/df34f344952740018d6948182a27ef8a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:12.420710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cb69f-8d18-4943-9af2-6fee28b08220 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.421229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cb69f-8d18-4943-9af2-6fee28b08220 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.421509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119cb69f-8d18-4943-9af2-6fee28b08220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.446596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-119cb69f-8d18-4943-9af2-6fee28b08220 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:12.448298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-119cb69f-8d18-4943-9af2-6fee28b08220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:12.470445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:12.471470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.471788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.472086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] PATH_INFO: `/v3/projects/2329bd36f61a497aabf6398da1883aa7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.473377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2329bd36f61a497aabf6398da1883aa7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:12.475497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:12.527266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 978/1946] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:12.529400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:12.535639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2329bd36f61a497aabf6398da1883aa7', 'request_id': 'req-9305368f-77b0-46f4-a208-6ae141a06b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.541005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.541303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.541734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.549903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2329bd36f61a497aabf6398da1883aa7', 'request_id': 'req-9305368f-77b0-46f4-a208-6ae141a06b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.561707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2329bd36f61a497aabf6398da1883aa7', 'request_id': 'req-9305368f-77b0-46f4-a208-6ae141a06b54'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.565913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:12.567141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:12.581820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] The token cache is being invalidate because project 2329bd36f61a497aabf6398da1883aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:12.582222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.583280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9305368f-77b0-46f4-a208-6ae141a06b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:12.586336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 969/1947] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:12 2022] DELETE /identity/v3/projects/2329bd36f61a497aabf6398da1883aa7 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:12.592885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.593288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.593811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.619612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:12.621136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:12.652098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 979/1948] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:12.658182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.675225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdb0a0-20c9-4876-a6bf-73002a15736e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:12.684873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:12.685653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.686075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.686482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.687244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:12.687614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:12.689355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102a0c66-9ebd-4d1c-90a9-049c8c7de09a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:12.702778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 970/1949] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:12.704227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 980/1950] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:12.708469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.716530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.718044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.720158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.731764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21975e0-f504-4385-87dc-4594d8ad7433 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:12.743237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:12.744388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.744977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.745500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.746249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:12.746888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:12.747569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:12.753280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:12.758195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 981/1951] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:12.762371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.777468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a95a9-5e9e-49ed-b0ae-1ed613807654 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:12.788228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:12.789150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:12.789576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:12.789970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:12.790616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:12.791007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:12.804844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:12.807791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 971/1952] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:12.814870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.833633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 982/1953] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:06:12.839253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:12.877641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642a2cc3-8cfd-429a-83a5-36fd5f979b81 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:12.898751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:13.007782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f3919-d540-4e83-8816-f6ccd5a6b2de admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.020314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.022035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.022335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.023767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.024691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.032239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.034498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.035412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.036803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.037314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.038470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.041168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.117403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 972/1954] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/users => generated 288 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:13.118437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 983/1955] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:12 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:13.119330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.119468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.155564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb4b2e-1cdc-48d3-8a1b-59c5cc4ea19a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.216408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762d075-6d45-4cad-a155-114c5767f1ae admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.247176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.249174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.251016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.251530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.253770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.259803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.281219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 984/1956] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:13.289295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:13.290442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.306068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb7c46-eb50-4f59-8146-11cece0ac617 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.318790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.319888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.324136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.325148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.327848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.348329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.416383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.417033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.417314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.417729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.418602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.421691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.436208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 985/1957] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:13.442146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.464417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d65759-db52-4d14-9ee9-b79d20a426a7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.471924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae135c-281d-49c8-a786-60e213711fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.473369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae135c-281d-49c8-a786-60e213711fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.473883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae135c-281d-49c8-a786-60e213711fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.474672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae135c-281d-49c8-a786-60e213711fbb admin admin] PATH_INFO: `/v3/projects/f6471a2bcc6248ab8ce1d3a6e37d162b/users/761f63413a484cf7aca920cfcb764c33/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.475592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 973/1958] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:13 2022] POST /identity/v3/users => generated 289 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:13.482549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.499242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae135c-281d-49c8-a786-60e213711fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6471a2bcc6248ab8ce1d3a6e37d162b, user_id=761f63413a484cf7aca920cfcb764c33, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.503914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae135c-281d-49c8-a786-60e213711fbb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.523377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e62e84-13a0-4525-969d-e5028c4d774a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.548067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 986/1959] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:13 2022] PUT /identity/v3/projects/f6471a2bcc6248ab8ce1d3a6e37d162b/users/761f63413a484cf7aca920cfcb764c33/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:13.554988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.555432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.555854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.608546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] MFA Rules not processed for user `761f63413a484cf7aca920cfcb764c33`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:13.610480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:13.694690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.695321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.695663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.695950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.696687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.698410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.699625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:13.716302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 974/1960] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:13.718113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.727713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 987/1961] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:13.742211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.745102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:13.745727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84c3b3-c4c8-4dc0-9511-4d9ae352a97d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.754955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.755761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.756142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.756468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.757198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.770255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.804752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 975/1962] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:13.809799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.814906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:13.852672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d19fe-b4c5-407a-b112-4f1ad62c1f5d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.859200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196d549-075a-444e-8cf1-0c1b7969cc9a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.862591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.863663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.864258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.864742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] PATH_INFO: `/v3/users/8a9c89b3718c4116bd045923e335b0a1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.866896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.867474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a9c89b3718c4116bd045923e335b0a1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.868044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.868597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.869072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.871369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.887836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.889924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:13.901869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 988/1963] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:13.902778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:13.934312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e780b-dfc1-41f7-82eb-8fbb7155c7ab service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:13.945662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b47f04c-d805-4301-93e4-67d62c2374c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:13.946592 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47f04c-d805-4301-93e4-67d62c2374c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:13.946884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47f04c-d805-4301-93e4-67d62c2374c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:13.947164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b47f04c-d805-4301-93e4-67d62c2374c7 admin admin] PATH_INFO: `/v3/projects/d1afa9bf432d4305a2a14d2fd158a803/users/ad09929f6a65444ab6fc6f682a6ab62d/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:13.979281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a9c89b3718c4116bd045923e335b0a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:13.979901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a9c89b3718c4116bd045923e335b0a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:13.989941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47f04c-d805-4301-93e4-67d62c2374c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1afa9bf432d4305a2a14d2fd158a803, user_id=ad09929f6a65444ab6fc6f682a6ab62d, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:13.993079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a9c89b3718c4116bd045923e335b0a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.000167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b47f04c-d805-4301-93e4-67d62c2374c7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.012589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a9c89b3718c4116bd045923e335b0a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.023401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7198c30e-3e84-44bd-a23a-2628cac678c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a9c89b3718c4116bd045923e335b0a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:14.031600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 976/1964] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:13 2022] DELETE /identity/v3/users/8a9c89b3718c4116bd045923e335b0a1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:14.037997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.039166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.039458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.040642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 989/1965] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:13 2022] PUT /identity/v3/projects/d1afa9bf432d4305a2a14d2fd158a803/users/ad09929f6a65444ab6fc6f682a6ab62d/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:14.053660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.054381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.055565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.074698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:14.075687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.112939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:14.117340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] MFA Rules not processed for user `ad09929f6a65444ab6fc6f682a6ab62d`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:14.120699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.194633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 977/1966] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:14.205633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.207746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.241891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 990/1967] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:14.248318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.252100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.275851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8632ca-5354-4cfe-8d50-b452146793f3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.283527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.284218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.284520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.284774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.285583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.287165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3e6de4-c63c-481a-a39f-cc1fe775fca1 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.287533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.292690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.293538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.293857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.294201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.308749 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.310333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.322955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 978/1968] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:14.330058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 991/1969] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/projects => generated 377 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:14.335084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.348259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4943f612-13f0-4af5-b4e4-d179d57da00c service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.458904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.460258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.460947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.461575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.464212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.469127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.520650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 979/1970] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:14.524844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.542526 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a2564-7acd-40e6-b3cc-abf3f5ab7bef admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.550974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.551991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.552509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.552968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.554063 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.557166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.579164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 992/1971] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:14.584709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.594503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcdfbd7-36b5-4584-b134-2e03eba2887b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.600129 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb8098-4720-4c15-b9a4-6ad87c5ba6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.600919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb8098-4720-4c15-b9a4-6ad87c5ba6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.601290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb8098-4720-4c15-b9a4-6ad87c5ba6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.601680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb8098-4720-4c15-b9a4-6ad87c5ba6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.602519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb8098-4720-4c15-b9a4-6ad87c5ba6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.604443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb8098-4720-4c15-b9a4-6ad87c5ba6a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.617879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 980/1972] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:14.622500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.635404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69373fa5-9d84-4a1a-b5db-db9695a5f30c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.641188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:14.642037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.642415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.642791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] PATH_INFO: `/v3/projects/dbe0612226654a4ca59e81414657f55f/users/2a8ed007102048ccbdd0d4140493bfd9/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.666950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbe0612226654a4ca59e81414657f55f, user_id=2a8ed007102048ccbdd0d4140493bfd9, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:14.670865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:14.692208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:14.698550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 993/1973] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:14 2022] PUT /identity/v3/projects/dbe0612226654a4ca59e81414657f55f/users/2a8ed007102048ccbdd0d4140493bfd9/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:14.712808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:14.713292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:14.713773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:14.753366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] MFA Rules not processed for user `2a8ed007102048ccbdd0d4140493bfd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:14.753366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.808217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:14.861289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.863168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:14.870109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 981/1974] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:14.876127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:14.992268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d060e10-2c31-48a9-8bd1-8af356d4065e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.997376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3b568-82e7-4696-9b8e-5bc1338f8c15 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:14.999598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.000511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.000766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.001138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.001881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.003959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.004434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.005410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.006007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.006431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] PATH_INFO: `/v3/projects/6e3c03ab94c741608e0d26f6d6161a97` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.008389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e3c03ab94c741608e0d26f6d6161a97)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.010638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.058880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 982/1975] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:14 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:15.065045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.097505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e3c03ab94c741608e0d26f6d6161a97', 'request_id': 'req-05a3e093-9c6d-49dd-919a-339d60943047'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.108000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e3c03ab94c741608e0d26f6d6161a97', 'request_id': 'req-05a3e093-9c6d-49dd-919a-339d60943047'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.117670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e3c03ab94c741608e0d26f6d6161a97', 'request_id': 'req-05a3e093-9c6d-49dd-919a-339d60943047'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.147455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] The token cache is being invalidate because project 6e3c03ab94c741608e0d26f6d6161a97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:15.148813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.149329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.151619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 994/1976] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:14 2022] DELETE /identity/v3/projects/6e3c03ab94c741608e0d26f6d6161a97 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:15.156614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.158276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.181296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbe958b-1ca8-4a48-a996-868e055eb8b7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.188816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.190315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.191286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.192417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.193907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.198023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.240654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 983/1977] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:15.244872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.247330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.266994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42606e0d-292f-4241-a6bf-34fe31679beb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.273501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.274287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.274682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.275050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.275850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.277842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.293530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 984/1978] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:15.299494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.305119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3e093-9c6d-49dd-919a-339d60943047 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.313264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.314385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.317369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.317369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] PATH_INFO: `/v3/users/d382d7ac4c2f49bba4e96d16381a5073` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.319443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d382d7ac4c2f49bba4e96d16381a5073)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.321171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82afa64-31ed-4818-ac10-018fa0a15722 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.321474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.328834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.329943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.330426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.330908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] PATH_INFO: `/v3/projects/6a5831346b8947448fb05bf8a145b8e5/users/431b479a85f14a8abfcb93d45f402fd4/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.359678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5831346b8947448fb05bf8a145b8e5, user_id=431b479a85f14a8abfcb93d45f402fd4, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.365067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.375314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:15.389759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 985/1979] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:15 2022] PUT /identity/v3/projects/6a5831346b8947448fb05bf8a145b8e5/users/431b479a85f14a8abfcb93d45f402fd4/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:15.394633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.406734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd382d7ac4c2f49bba4e96d16381a5073'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.432037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:15.432357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd382d7ac4c2f49bba4e96d16381a5073'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.444107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd382d7ac4c2f49bba4e96d16381a5073'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.460576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd382d7ac4c2f49bba4e96d16381a5073'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.461345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd382d7ac4c2f49bba4e96d16381a5073'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:15.468813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 995/1980] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:15 2022] DELETE /identity/v3/users/d382d7ac4c2f49bba4e96d16381a5073 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:15.485165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35798f9c-d757-4a2e-a625-dd619c509d0d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.563572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab06b-c56c-4583-927c-e3e62d29dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.564255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab06b-c56c-4583-927c-e3e62d29dc8a admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.564544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab06b-c56c-4583-927c-e3e62d29dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.564813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab06b-c56c-4583-927c-e3e62d29dc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.565559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ab06b-c56c-4583-927c-e3e62d29dc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.567572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6ab06b-c56c-4583-927c-e3e62d29dc8a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.587694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.590513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.592265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 986/1981] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:15.609316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc59646-1dec-4beb-997d-723345e78b88 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.616836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.617970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.619043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.619043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] PATH_INFO: `/v3/projects/6a5831346b8947448fb05bf8a145b8e5/users/431b479a85f14a8abfcb93d45f402fd4/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.621987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5831346b8947448fb05bf8a145b8e5, user_id=431b479a85f14a8abfcb93d45f402fd4, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.625035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.637842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 996/1982] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:15 2022] PUT /identity/v3/projects/6a5831346b8947448fb05bf8a145b8e5/users/431b479a85f14a8abfcb93d45f402fd4/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:15.650747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.650747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.650747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.708390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] MFA Rules not processed for user `431b479a85f14a8abfcb93d45f402fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:15.710334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.822507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.824620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.829431 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 987/1983] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:15 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:15.843540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:15.845399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:15.914677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0804ae-6ac0-45b4-9eb9-09f99dfdd0e9 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.917842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:15.920737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa648b46-588c-4553-9865-483381e721af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.921424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa648b46-588c-4553-9865-483381e721af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.921700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa648b46-588c-4553-9865-483381e721af admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.921968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa648b46-588c-4553-9865-483381e721af admin admin] PATH_INFO: `/v3/projects/b5ca51cc61094865a08aa74d316802a8` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.923362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa648b46-588c-4553-9865-483381e721af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ca51cc61094865a08aa74d316802a8)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:15.925704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa648b46-588c-4553-9865-483381e721af admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:15.927042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb36af-7d8b-4eb0-a92d-590235a77478 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:15.931317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:15.932277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:15.932561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:15.932877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:15.934787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:16.008480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-aa648b46-588c-4553-9865-483381e721af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ca51cc61094865a08aa74d316802a8', 'request_id': 'req-aa648b46-588c-4553-9865-483381e721af'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:16.018691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-aa648b46-588c-4553-9865-483381e721af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ca51cc61094865a08aa74d316802a8', 'request_id': 'req-aa648b46-588c-4553-9865-483381e721af'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:16.026363 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-aa648b46-588c-4553-9865-483381e721af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ca51cc61094865a08aa74d316802a8', 'request_id': 'req-aa648b46-588c-4553-9865-483381e721af'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:16.027886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.031462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abca44-4311-4b2b-babc-6035fe04fba6 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.042852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-aa648b46-588c-4553-9865-483381e721af admin admin] The token cache is being invalidate because project b5ca51cc61094865a08aa74d316802a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:16.043160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-aa648b46-588c-4553-9865-483381e721af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:16.043846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-aa648b46-588c-4553-9865-483381e721af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:16.046559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 997/1984] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:15 2022] DELETE /identity/v3/projects/b5ca51cc61094865a08aa74d316802a8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:16.051157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 988/1985] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:16.052237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.052528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.052891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.078343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:16.079658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:16.111132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:16.178166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 998/1986] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:16.182940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.183406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.183782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.218094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:16.220486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:16.236723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 989/1987] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:16.242685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:16.253717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13740f0-8f9c-4a16-bf3c-db3161c77acd None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:16.259292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:16.259920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.260234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.260507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.263535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:16.263535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.274167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.303024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 999/1988] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:06:16.308434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:16.334640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e5cfa-c241-497c-ad57-3bf92d4da309 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:16.355226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:16.356397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.356808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.357779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.359172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.363553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.415898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 990/1989] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:16 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:16.418467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:16.443126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758cab1-edb9-4f01-9f7e-c37a3e89fd18 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:16.527163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:16.527758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.528150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.528493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.529965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.531889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.570022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1000/1990] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:16 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:16.571680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:16.588058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444a9d9-5f9d-4e5f-b142-a2f66d3732fd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:16.601434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:16.602157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.603144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.603502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.604510 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.608271 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.661659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 991/1991] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:16.662156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:16.677894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92790de7-eb35-4c32-ba2f-cfa5b60bdf21 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:16.687902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621e771-bf56-4b44-bac9-73cab2971231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:16.688593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e771-bf56-4b44-bac9-73cab2971231 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.688880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e771-bf56-4b44-bac9-73cab2971231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.689159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621e771-bf56-4b44-bac9-73cab2971231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.689824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e771-bf56-4b44-bac9-73cab2971231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.691561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621e771-bf56-4b44-bac9-73cab2971231 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.727894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1001/1992] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:16.733319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:16.753103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37985dd-1762-4671-aeef-6292c7ecc1ca admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:16.761083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:16.762411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.762943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.763445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] PATH_INFO: `/v3/projects/08035111d1104ac8a68e36845fab2fed/users/40e44bc9e04544be933163aae190a754/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.811691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08035111d1104ac8a68e36845fab2fed, user_id=40e44bc9e04544be933163aae190a754, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:16.815652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:16.858549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 992/1993] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:16 2022] PUT /identity/v3/projects/08035111d1104ac8a68e36845fab2fed/users/40e44bc9e04544be933163aae190a754/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:16.861437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:16.862465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:16.862597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:16.923610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:16.926512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:17.057279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1002/1994] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:17.064927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:17.068264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:17.148837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36987754-f95c-46b1-8a9a-dd793f26e252 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:17.152995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb31a2-a71c-41ba-a9ea-7cc5a60535b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:17.153659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb31a2-a71c-41ba-a9ea-7cc5a60535b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:17.154774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb31a2-a71c-41ba-a9ea-7cc5a60535b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:17.155058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb31a2-a71c-41ba-a9ea-7cc5a60535b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:17.166077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb31a2-a71c-41ba-a9ea-7cc5a60535b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:17.167639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb31a2-a71c-41ba-a9ea-7cc5a60535b6 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:17.184622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 993/1995] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:18.332506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:18.336348 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:18.436204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:18.517535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:18.518305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:18.518305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:18.518965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52eb781-8b04-4389-947c-f60eb3ccce86 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:18.529942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:18.530255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:18.532439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:18.533416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:18.537568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:18.559750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] MFA Rules not processed for user `40e44bc9e04544be933163aae190a754`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:18.561861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:18.600942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:18.677805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 994/1996] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:18.682420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:18.697019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43bdb4-3f4e-4773-865d-e22d1e1b5827 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:18.704188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:18.704828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:18.705109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:18.705386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:18.706200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:18.708164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:18.714485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:18.722257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:18.757509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 995/1997] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:18 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:18.758916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:18.766713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1003/1998] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:18.977610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533642e-e319-47d7-badb-79dc7415e1a1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:18.993020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:18.994647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:18.995363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:18.996775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:19.002091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:19.009986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:19.073905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 996/1999] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:18 2022] POST /identity/v3/users => generated 292 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:19.075824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:19.095737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9849916d-bedb-4ad3-bf42-d182b10627e8 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:19.104046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:19.105322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:19.105820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:19.106314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:19.108125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:19.111240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:19.137238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1004/2000] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:19.139443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:19.162289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede656f-83ff-41d3-adfd-f20d70597826 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:19.180646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:19.181677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:19.182098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:19.182483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] PATH_INFO: `/v3/projects/d09e74b746e34fd68314219faad6b428/users/e0acdf96782b45d39a4cc1c20188ac9b/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:19.229623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09e74b746e34fd68314219faad6b428, user_id=e0acdf96782b45d39a4cc1c20188ac9b, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:19.234079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:19.289482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 997/2001] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:19 2022] PUT /identity/v3/projects/d09e74b746e34fd68314219faad6b428/users/e0acdf96782b45d39a4cc1c20188ac9b/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:19.295876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:19.314768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237dda1-1daa-4958-ac46-bdfbea505883 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:19.513103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:19.515122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:19.516600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:19.517054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:19.518083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:19.520628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:19.540634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1005/2002] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:19.546852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:19.562141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca4786-7d1f-4e5c-bd6a-d96e150f971e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:19.569247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab9633c-8db3-47ea-9171-74802739af09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:19.570314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9633c-8db3-47ea-9171-74802739af09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:19.570763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9633c-8db3-47ea-9171-74802739af09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:19.571194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab9633c-8db3-47ea-9171-74802739af09 admin admin] PATH_INFO: `/v3/projects/d09e74b746e34fd68314219faad6b428/users/e0acdf96782b45d39a4cc1c20188ac9b/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:19.573458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9633c-8db3-47ea-9171-74802739af09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d09e74b746e34fd68314219faad6b428, user_id=e0acdf96782b45d39a4cc1c20188ac9b, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:19.576687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab9633c-8db3-47ea-9171-74802739af09 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:19.593337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 998/2003] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:19 2022] PUT /identity/v3/projects/d09e74b746e34fd68314219faad6b428/users/e0acdf96782b45d39a4cc1c20188ac9b/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:20.385355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:20.387313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:20.572093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3a629b-952b-4e03-af9a-c5538bbb7bce admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:20.637103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:20.683274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:20.684046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:20.684410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:20.684686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:20.691991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:20.852688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:20.854745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:20.881417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1006/2004] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:21.208478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:21.209033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:21.209243 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:21.271371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] MFA Rules not processed for user `e0acdf96782b45d39a4cc1c20188ac9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:21.272730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:21.333684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:21.373528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 999/2005] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:21.387529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:21.414021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ce8d3-f315-40c5-9193-4421f142d22e service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:21.422632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709d187-e579-43fe-9548-39fab9bccdaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:21.423488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d187-e579-43fe-9548-39fab9bccdaf service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:21.425246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d187-e579-43fe-9548-39fab9bccdaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:21.425947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709d187-e579-43fe-9548-39fab9bccdaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:21.531662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709d187-e579-43fe-9548-39fab9bccdaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:21.533794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709d187-e579-43fe-9548-39fab9bccdaf service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:21.563499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1007/2006] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:21.599455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:21.601641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:21.703023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4668f5-2ea3-4ec4-b4dc-9c90cf099025 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:21.713785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:21.714788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:21.715264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:21.715697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] PATH_INFO: `/v3/users/761f63413a484cf7aca920cfcb764c33` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:21.718612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=761f63413a484cf7aca920cfcb764c33)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:21.722702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:21.798037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761f63413a484cf7aca920cfcb764c33'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:21.799273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761f63413a484cf7aca920cfcb764c33'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:21.809730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761f63413a484cf7aca920cfcb764c33'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:21.829194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761f63413a484cf7aca920cfcb764c33'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:21.847987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761f63413a484cf7aca920cfcb764c33'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:21.856601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1000/2007] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:21 2022] DELETE /identity/v3/users/761f63413a484cf7aca920cfcb764c33 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:21.863342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:21.863897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:21.863897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:21.891478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:21.892762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:22.019322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1008/2008] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:22.032566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:22.033977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:22.126945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db346142-b1de-4a5d-aa60-f210e1e6cbce admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:22.134996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462b6e4-490f-4aaa-8587-a565e4c1e2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:22.135990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462b6e4-490f-4aaa-8587-a565e4c1e2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:22.136456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462b6e4-490f-4aaa-8587-a565e4c1e2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:22.136882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462b6e4-490f-4aaa-8587-a565e4c1e2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:22.153118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462b6e4-490f-4aaa-8587-a565e4c1e2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:22.155320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d462b6e4-490f-4aaa-8587-a565e4c1e2a0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:22.167553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1001/2009] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:22.735723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:22.737495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:22.751765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16477c-c680-46e5-8d21-d5b9769be3bc None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:22.757853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:22.758746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:22.759041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:22.759354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] PATH_INFO: `/v3/projects/f6471a2bcc6248ab8ce1d3a6e37d162b` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:22.760716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6471a2bcc6248ab8ce1d3a6e37d162b)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:22.762807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:22.791698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:22.859132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6471a2bcc6248ab8ce1d3a6e37d162b', 'request_id': 'req-52cddcb5-1a2b-447b-97d9-5422ab54af08'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:22.868968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6471a2bcc6248ab8ce1d3a6e37d162b', 'request_id': 'req-52cddcb5-1a2b-447b-97d9-5422ab54af08'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:22.878106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6471a2bcc6248ab8ce1d3a6e37d162b', 'request_id': 'req-52cddcb5-1a2b-447b-97d9-5422ab54af08'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:22.896373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] The token cache is being invalidate because project f6471a2bcc6248ab8ce1d3a6e37d162b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:22.896711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:22.897670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-52cddcb5-1a2b-447b-97d9-5422ab54af08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:22.899717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1009/2010] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:22 2022] DELETE /identity/v3/projects/f6471a2bcc6248ab8ce1d3a6e37d162b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:22.906070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:22.906070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:22.909284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:22.928819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:22.940901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:22.944303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:23.103321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1002/2011] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:23.107759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.109263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.109676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.140810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:23.142602 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:23.154673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1010/2012] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:23.160877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.178948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e603fc-911a-4e12-91a6-cd6aff7c69cb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.186915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.187822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.188391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.188782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.189905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:23.190373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.193302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.214446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1003/2013] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:06:23.220598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.236348 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2462e4-4f08-4e4b-a86e-671d3bacc77c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.244217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.245034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.245481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.245850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.246988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.250673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.289449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1011/2014] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:23 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:23.293564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.312266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a020436-3507-438a-a70f-8558647c940e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.410643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.411739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.412305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.412807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.414044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.417835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.457784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1004/2015] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:23 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:23.461774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.477441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9ecf8-6647-44c5-af1d-f3cba5e60b12 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.492695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.493336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.493622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.493896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.494570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.496269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.512056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1012/2016] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:23.515582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.536700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da839ae-a564-4ccb-a9d3-927c22587f17 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.541901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22827c-df92-477d-a19d-5367f494aac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.542606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22827c-df92-477d-a19d-5367f494aac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.543030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22827c-df92-477d-a19d-5367f494aac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.543396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22827c-df92-477d-a19d-5367f494aac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.544299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22827c-df92-477d-a19d-5367f494aac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.546838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22827c-df92-477d-a19d-5367f494aac1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.564050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1005/2017] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:23.567945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.582957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be40776-3c54-4da6-9aef-a6a78764a654 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.589954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.590864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.591864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.592233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] PATH_INFO: `/v3/projects/b2365f1500c5405aab2ea6e892fbb832/users/fe027aad062342688c587428f2bfbfe7/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.613844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2365f1500c5405aab2ea6e892fbb832, user_id=fe027aad062342688c587428f2bfbfe7, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.617736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.636255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1013/2018] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:23 2022] PUT /identity/v3/projects/b2365f1500c5405aab2ea6e892fbb832/users/fe027aad062342688c587428f2bfbfe7/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:23.642983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.643529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.644066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.675446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:23.675446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:23.801668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1006/2019] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:23.816151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:23.821794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:23.900436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b932e70-72d9-47c8-84e0-8b8dea1c66b2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:23.906016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d3f1-0462-466b-9a46-d09aa4781015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:23.907224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d3f1-0462-466b-9a46-d09aa4781015 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:23.907703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d3f1-0462-466b-9a46-d09aa4781015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:23.907814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d3f1-0462-466b-9a46-d09aa4781015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:23.921150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5d3f1-0462-466b-9a46-d09aa4781015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:23.924536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5d3f1-0462-466b-9a46-d09aa4781015 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:23.942522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1014/2020] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:25.246945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:25.249532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:25.270851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:25.351117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e0deb4-3cf1-4450-a4dd-1875fbd4bc15 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:25.363257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:25.364341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:25.366366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:25.366820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:25.369926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:25.518400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab091de-a6bc-4326-b8f3-187801e1cf11 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:25.519278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab091de-a6bc-4326-b8f3-187801e1cf11 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:25.519736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab091de-a6bc-4326-b8f3-187801e1cf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:25.549057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:25.553894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:25.596635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1007/2021] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:25.639849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fab091de-a6bc-4326-b8f3-187801e1cf11 None None] MFA Rules not processed for user `fe027aad062342688c587428f2bfbfe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:25.641536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fab091de-a6bc-4326-b8f3-187801e1cf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:25.715717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:25.812721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1015/2022] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:25.933267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:25.958149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca6485b-8e46-40af-a19b-8d878b29adf9 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:25.964737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:25.965413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:25.965695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:25.965998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:25.977143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:25.978595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:25.993119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1008/2023] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:27.805428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559b86f-0e31-4e49-8e6b-fd2cfe3103d8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:27.806428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559b86f-0e31-4e49-8e6b-fd2cfe3103d8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:27.806895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559b86f-0e31-4e49-8e6b-fd2cfe3103d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:27.828648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-9559b86f-0e31-4e49-8e6b-fd2cfe3103d8 None None] MFA Rules not processed for user `fe027aad062342688c587428f2bfbfe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:27.830114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9559b86f-0e31-4e49-8e6b-fd2cfe3103d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:27.846335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1016/2024] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:29.620361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:29.622852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:29.644127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf494e-0089-4daf-aa3a-d1435c92997c service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:29.649681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:29.650322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:29.651370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:29.651704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] PATH_INFO: `/v3/users/ad09929f6a65444ab6fc6f682a6ab62d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:29.654525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad09929f6a65444ab6fc6f682a6ab62d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:29.656522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:29.769046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad09929f6a65444ab6fc6f682a6ab62d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:29.772127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad09929f6a65444ab6fc6f682a6ab62d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:29.792098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad09929f6a65444ab6fc6f682a6ab62d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:29.809089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad09929f6a65444ab6fc6f682a6ab62d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:29.819794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad09929f6a65444ab6fc6f682a6ab62d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:29.824373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1009/2025] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:29 2022] DELETE /identity/v3/users/ad09929f6a65444ab6fc6f682a6ab62d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:29.834652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:29.835276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:29.835459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:29.857115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:29.858759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:29.860881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:29.863014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:29.902875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:29.964738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037a2ea-98cd-4871-b19c-8499beb76445 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:29.971298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1017/2026] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:29.971595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:29.973557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:29.973557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:29.973951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] PATH_INFO: `/v3/users/46f92714bf894f3ab5d60a2240dd5068` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:29.975383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46f92714bf894f3ab5d60a2240dd5068)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:29.977730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:29.978705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:29.980558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:30.050954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46f92714bf894f3ab5d60a2240dd5068'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.052186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46f92714bf894f3ab5d60a2240dd5068'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.060593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3365bc2-db7d-44df-bced-80452c4f0b79 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:30.066026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d6e37-c6b6-40e9-85d4-41a99d6586ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:30.066679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d6e37-c6b6-40e9-85d4-41a99d6586ee service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:30.066942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d6e37-c6b6-40e9-85d4-41a99d6586ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:30.067425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d6e37-c6b6-40e9-85d4-41a99d6586ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:30.068074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46f92714bf894f3ab5d60a2240dd5068'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.079102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d6e37-c6b6-40e9-85d4-41a99d6586ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:30.081380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d6e37-c6b6-40e9-85d4-41a99d6586ee service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:30.091679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46f92714bf894f3ab5d60a2240dd5068'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.092456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1018/2027] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:30.105545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46f92714bf894f3ab5d60a2240dd5068'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.112788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1010/2028] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:29 2022] DELETE /identity/v3/users/46f92714bf894f3ab5d60a2240dd5068 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:30.117786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:30.118072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:30.118351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:30.140680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:30.141942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:30.214843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1019/2029] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:30.225443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:30.227203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:30.336889 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4b09f-3871-44af-bb17-79151a1b4d98 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:30.342792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:30.343428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:30.343715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:30.343991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:30.359760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:30.363884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:30.375660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1011/2030] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:30.585095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:30.588303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:30.621301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72f1e2-27ab-49b0-a0f7-9bd3023b160f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:30.630250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:30.631079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:30.631524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:30.631933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] PATH_INFO: `/v3/projects/d1afa9bf432d4305a2a14d2fd158a803` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:30.633500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1afa9bf432d4305a2a14d2fd158a803)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:30.635558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:30.637629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:30.704217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1afa9bf432d4305a2a14d2fd158a803', 'request_id': 'req-a0864f5a-7f1e-49b7-abed-422b3025e332'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.711633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1afa9bf432d4305a2a14d2fd158a803', 'request_id': 'req-a0864f5a-7f1e-49b7-abed-422b3025e332'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.719347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1afa9bf432d4305a2a14d2fd158a803', 'request_id': 'req-a0864f5a-7f1e-49b7-abed-422b3025e332'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.738115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] The token cache is being invalidate because project d1afa9bf432d4305a2a14d2fd158a803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:30.738434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.739091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.741090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1020/2031] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:30 2022] DELETE /identity/v3/projects/d1afa9bf432d4305a2a14d2fd158a803 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:30.745395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:30.755499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccee861-4b99-4695-9e4c-7703ed03fde6 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:30.811684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:30.833655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:30.834265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:30.834583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:30.834847 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] PATH_INFO: `/v3/users/2a8ed007102048ccbdd0d4140493bfd9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:30.836160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8ed007102048ccbdd0d4140493bfd9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:30.843963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:30.945275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8ed007102048ccbdd0d4140493bfd9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.946092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8ed007102048ccbdd0d4140493bfd9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.959719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8ed007102048ccbdd0d4140493bfd9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.981306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8ed007102048ccbdd0d4140493bfd9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:30.996362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8fd70dad-7d4e-44ab-acd7-aca5ece36b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8ed007102048ccbdd0d4140493bfd9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.007259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1012/2032] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:30 2022] DELETE /identity/v3/users/2a8ed007102048ccbdd0d4140493bfd9 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:31.085998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.087758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:31.125203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0864f5a-7f1e-49b7-abed-422b3025e332 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.249837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.250343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.262390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.262390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] PATH_INFO: `/v3/projects/7fc65b04013f453c99dde9d2ae73889e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.262390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fc65b04013f453c99dde9d2ae73889e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.262390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.392667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fc65b04013f453c99dde9d2ae73889e', 'request_id': 'req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.405279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fc65b04013f453c99dde9d2ae73889e', 'request_id': 'req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.416591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fc65b04013f453c99dde9d2ae73889e', 'request_id': 'req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.442010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] The token cache is being invalidate because project 7fc65b04013f453c99dde9d2ae73889e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:31.442643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.443629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.446202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1021/2033] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:31 2022] DELETE /identity/v3/projects/7fc65b04013f453c99dde9d2ae73889e => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:31.451861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ddc0-1cda-4e2a-a68c-a8389cbeeaea None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.451861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ddc0-1cda-4e2a-a68c-a8389cbeeaea None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.452207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74ddc0-1cda-4e2a-a68c-a8389cbeeaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.479683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cf74ddc0-1cda-4e2a-a68c-a8389cbeeaea None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:31.480980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cf74ddc0-1cda-4e2a-a68c-a8389cbeeaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:31.500299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.502372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:31.556878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:31.569108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1013/2034] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:31.577715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.578018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.578278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.613407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:31.615508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:31.622700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f23e1-a310-4799-a34d-6da4f9e42e3c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.628838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.629504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.629831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.630110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] PATH_INFO: `/v3/projects/dbe0612226654a4ca59e81414657f55f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.630987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1014/2035] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:31.631446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbe0612226654a4ca59e81414657f55f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.633629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.638035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.654928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc4708-5a9f-412a-a170-43d617d40cfb None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.665349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.666311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.667113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.667344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.669151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:31.669726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.672596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.693730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1015/2036] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:31.696809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.713057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba35cf-6d23-4261-9fb9-be218cd0dc07 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.720902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.721850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.722142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.722476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.723390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.726042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.763107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbe0612226654a4ca59e81414657f55f', 'request_id': 'req-f8801753-0744-4345-9f2e-b5ea7daa739a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.771551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbe0612226654a4ca59e81414657f55f', 'request_id': 'req-f8801753-0744-4345-9f2e-b5ea7daa739a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.777561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1016/2037] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:31 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:31.778863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbe0612226654a4ca59e81414657f55f', 'request_id': 'req-f8801753-0744-4345-9f2e-b5ea7daa739a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.782384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.798258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] The token cache is being invalidate because project dbe0612226654a4ca59e81414657f55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:31.798845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.799706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:31.801674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1022/2038] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:31 2022] DELETE /identity/v3/projects/dbe0612226654a4ca59e81414657f55f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:31.805516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.806946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:31.826594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:31.878150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65841c86-0101-4921-8c3d-a489973242e8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.883392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.884044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.884349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.884652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.885259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.887226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.910178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1017/2039] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:31 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:31.914639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.917279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:31.932955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8801753-0744-4345-9f2e-b5ea7daa739a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.933535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f4c2c-d06a-4a91-8817-6ed6d425d495 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.941197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.941197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.941197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.941771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.942261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.943969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.950761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.953132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.953717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.954276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] PATH_INFO: `/v3/users/431b479a85f14a8abfcb93d45f402fd4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.956606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=431b479a85f14a8abfcb93d45f402fd4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.959259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:31.963375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1018/2040] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:31.964668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:31.979790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746db9d-4e2c-4975-93aa-8c8cc81f7fbb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:31.986017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:31.986830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:31.987033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:31.987426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:31.988693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:31.990420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:32.008134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1019/2041] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:32.009736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:32.025357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810cc36d-3c67-4ba4-ab09-953bd59a5b5b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:32.032042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a7631-e0c7-4bed-979f-3a6094ccdaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:32.033103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a7631-e0c7-4bed-979f-3a6094ccdaf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:32.033426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a7631-e0c7-4bed-979f-3a6094ccdaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:32.033830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a7631-e0c7-4bed-979f-3a6094ccdaf2 admin admin] PATH_INFO: `/v3/projects/6e4af9139d2b47cf8ea8646c5a4f4a9f/users/74fa7242b82941cc9ef9f21f122b1e4c/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:32.050926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431b479a85f14a8abfcb93d45f402fd4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:32.061577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:32.064364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431b479a85f14a8abfcb93d45f402fd4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:32.064773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a7631-e0c7-4bed-979f-3a6094ccdaf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4af9139d2b47cf8ea8646c5a4f4a9f, user_id=74fa7242b82941cc9ef9f21f122b1e4c, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:32.069071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a7631-e0c7-4bed-979f-3a6094ccdaf2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:32.079659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431b479a85f14a8abfcb93d45f402fd4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:32.093870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1020/2042] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:32 2022] PUT /identity/v3/projects/6e4af9139d2b47cf8ea8646c5a4f4a9f/users/74fa7242b82941cc9ef9f21f122b1e4c/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:32.099648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:32.100615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:32.101476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:32.102165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431b479a85f14a8abfcb93d45f402fd4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:32.104769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431b479a85f14a8abfcb93d45f402fd4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:32.115048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1023/2043] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:31 2022] DELETE /identity/v3/users/431b479a85f14a8abfcb93d45f402fd4 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:32.139877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:32.141292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:32.296709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1021/2044] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:32.309467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:32.313336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:32.503407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569289ab-446f-4045-b5e7-1b78a1a162c2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:32.519399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951aba27-76a0-407e-bb89-300002d4a0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:32.521385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aba27-76a0-407e-bb89-300002d4a0dd service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:32.521688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aba27-76a0-407e-bb89-300002d4a0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:32.523991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951aba27-76a0-407e-bb89-300002d4a0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:32.552287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951aba27-76a0-407e-bb89-300002d4a0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:32.554317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951aba27-76a0-407e-bb89-300002d4a0dd service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:32.599447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1024/2045] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:32.894854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:32.897184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:32.937871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9233c-051c-4cb9-9b2b-318b6a129c75 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:32.952551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b636-6581-4479-9c11-98716282562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:32.954962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b636-6581-4479-9c11-98716282562a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:32.955109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b636-6581-4479-9c11-98716282562a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:32.956961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b636-6581-4479-9c11-98716282562a admin admin] PATH_INFO: `/v3/projects/6a5831346b8947448fb05bf8a145b8e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:32.959747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b636-6581-4479-9c11-98716282562a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5831346b8947448fb05bf8a145b8e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:32.965587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b636-6581-4479-9c11-98716282562a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.101143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c388b636-6581-4479-9c11-98716282562a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5831346b8947448fb05bf8a145b8e5', 'request_id': 'req-c388b636-6581-4479-9c11-98716282562a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.110784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c388b636-6581-4479-9c11-98716282562a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5831346b8947448fb05bf8a145b8e5', 'request_id': 'req-c388b636-6581-4479-9c11-98716282562a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.116174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:33.119237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c388b636-6581-4479-9c11-98716282562a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5831346b8947448fb05bf8a145b8e5', 'request_id': 'req-c388b636-6581-4479-9c11-98716282562a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.136596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c388b636-6581-4479-9c11-98716282562a admin admin] The token cache is being invalidate because project 6a5831346b8947448fb05bf8a145b8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:33.136897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c388b636-6581-4479-9c11-98716282562a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.138488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c388b636-6581-4479-9c11-98716282562a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:33.144056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1022/2046] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:32 2022] DELETE /identity/v3/projects/6a5831346b8947448fb05bf8a145b8e5 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:33.147504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.148075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.148673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.178297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:33.179588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:33.261855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1025/2047] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:33.272535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.273089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.273590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.300351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:33.301963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:33.317230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1023/2048] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:33.325244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.342863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b34dd-a286-4898-88f8-979d7d0958ae None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.360115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.361050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.361490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.361921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.362748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:33.363211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.365127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.385329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1026/2049] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:33.386397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.415277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b3d29-3216-446d-879f-0db8397ea77f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.426599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.427707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.428248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.428733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.429972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.436039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.497704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1024/2050] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:33 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:33.499446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.510797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd088dc-f9a7-4f23-a63f-d1c7a7dea765 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.582537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.583846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.584325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.584760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.585911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.588033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.598831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:33.619098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1027/2051] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:33 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:33.621901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.636867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96ace5-a1d1-4977-af2d-b8fe10459949 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.644795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.646040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.646549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.647050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.648292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.651480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.674274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1025/2052] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:33.679852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.696330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955e074-194d-4745-8b8e-1f6598c80878 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.702621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564cb72-0cf1-4087-a6fb-d8493824d403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.703659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564cb72-0cf1-4087-a6fb-d8493824d403 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.704226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564cb72-0cf1-4087-a6fb-d8493824d403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.704735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564cb72-0cf1-4087-a6fb-d8493824d403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.705956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564cb72-0cf1-4087-a6fb-d8493824d403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.709159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564cb72-0cf1-4087-a6fb-d8493824d403 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.730147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1028/2053] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:33.740862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:33.753516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19563691-e3ce-42a2-aa20-2c0e32ddc843 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:33.761140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:33.761838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.762212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.762981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] PATH_INFO: `/v3/projects/a3498c2154f24e2f9bcca4da53e2f705/users/b6e438a0e7064be4aa3e19ccbe595cb7/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.788953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3498c2154f24e2f9bcca4da53e2f705, user_id=b6e438a0e7064be4aa3e19ccbe595cb7, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:33.792165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:33.820314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1026/2054] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:33 2022] PUT /identity/v3/projects/a3498c2154f24e2f9bcca4da53e2f705/users/b6e438a0e7064be4aa3e19ccbe595cb7/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:33.844426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:33.845279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:33.846195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:33.938789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] MFA Rules not processed for user `b6e438a0e7064be4aa3e19ccbe595cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:33.939391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:34.078171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1029/2055] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:34.085443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.302611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ee124-84a1-4d9b-8e21-774b7a110318 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.321882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.322958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.323511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.326555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.326555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.334003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.404948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1027/2056] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:34 2022] POST /identity/v3/projects => generated 385 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:34.405222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.424117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526ae06-1e55-4369-ae4f-478e8d621da0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.572237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.573171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.573524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.573910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.574690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.579623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.639156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1030/2057] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:34 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:34.643667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.667816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa914fc-2ee7-4843-b559-12d19c8a2223 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.691785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.692458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.692738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.693012 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.698627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.700739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.749760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.751059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1028/2058] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:34.767038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c604b-7db3-4c4f-8a1c-776737047e57 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.781144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b41d9-ca3c-4b95-9599-1780379d0586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.781962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b41d9-ca3c-4b95-9599-1780379d0586 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.782403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b41d9-ca3c-4b95-9599-1780379d0586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.782794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b41d9-ca3c-4b95-9599-1780379d0586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.785945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b41d9-ca3c-4b95-9599-1780379d0586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.794921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b41d9-ca3c-4b95-9599-1780379d0586 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.810578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1031/2059] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:34.814552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:34.829517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087c5a8-32e6-4345-8beb-3ec0e4e92ed0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:34.852197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:34.852977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.853197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.853592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] PATH_INFO: `/v3/projects/bd6291120b6740d48f087f05eabbae16/users/afce61dcfabf460c9c8e31bd6f9f0041/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.884801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd6291120b6740d48f087f05eabbae16, user_id=afce61dcfabf460c9c8e31bd6f9f0041, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:34.886747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:34.910452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1029/2060] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:34 2022] PUT /identity/v3/projects/bd6291120b6740d48f087f05eabbae16/users/afce61dcfabf460c9c8e31bd6f9f0041/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:34.921919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:34.922239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:34.922481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:34.956880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] MFA Rules not processed for user `afce61dcfabf460c9c8e31bd6f9f0041`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:34.958147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:34.997734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:35.066329 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.068058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1032/2061] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:35.157947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:35.201095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5183aeb-fd51-40ee-8ba2-c47fbcd9aed9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.214378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.216063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.216590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.217111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.218337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.222335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.280059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1030/2062] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:35 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:35.282421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.298788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d7cf5-1cd0-4d7b-8af2-898af65b644b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.439622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.442030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.446367 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.450422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.451258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.454805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.492209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1033/2063] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:35 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:35.499373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.518430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1a5a0-c192-454a-b047-c06d7e99a749 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.526123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.527342 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.528676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.529296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.534620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.537248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.556947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1031/2064] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:35.562304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.588597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513c813-4e9b-487c-82d0-2a89f435e519 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.601713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.602724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.603262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.603600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] PATH_INFO: `/v3/projects/f8d3fd75cd5c4f04ad4a10de6e0f3f9f/users/355a960670e140dab1c17e7e6e33c48f/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.629565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d3fd75cd5c4f04ad4a10de6e0f3f9f, user_id=355a960670e140dab1c17e7e6e33c48f, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.632613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.668076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1034/2065] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:35 2022] PUT /identity/v3/projects/f8d3fd75cd5c4f04ad4a10de6e0f3f9f/users/355a960670e140dab1c17e7e6e33c48f/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:35.673824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.685814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6a31c-d3b9-4dd4-aec8-4ba105129853 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.769221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb3b5c-ffb4-4455-b725-69eafb6c04cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.769872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb3b5c-ffb4-4455-b725-69eafb6c04cf admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.770156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb3b5c-ffb4-4455-b725-69eafb6c04cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.770435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb3b5c-ffb4-4455-b725-69eafb6c04cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.771125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb3b5c-ffb4-4455-b725-69eafb6c04cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.778600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb3b5c-ffb4-4455-b725-69eafb6c04cf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.792719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1032/2066] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:35.799095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:35.816492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a532c9-ce99-471e-92af-67c856c98966 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:35.824939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:35.829317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.829553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.829796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] PATH_INFO: `/v3/projects/f8d3fd75cd5c4f04ad4a10de6e0f3f9f/users/355a960670e140dab1c17e7e6e33c48f/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.834269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d3fd75cd5c4f04ad4a10de6e0f3f9f, user_id=355a960670e140dab1c17e7e6e33c48f, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:35.841655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:35.886738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1035/2067] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:35 2022] PUT /identity/v3/projects/f8d3fd75cd5c4f04ad4a10de6e0f3f9f/users/355a960670e140dab1c17e7e6e33c48f/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:35.889041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:35.889525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:35.889809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:35.930971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] MFA Rules not processed for user `355a960670e140dab1c17e7e6e33c48f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:35.932470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:36.145739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:36.146691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1033/2068] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:36.147846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:36.284218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7042a-c278-42f0-b1d2-cd621a8c1d26 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:36.308531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:36.308531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:36.308531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:36.308531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:36.391532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:36.419003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:36.421721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:36.436939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1036/2069] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:38.246771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:38.257974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d92fb5-0b6b-47a1-ba51-a5f7e8cc1353 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:38.263525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ae8e8-61c5-45e3-b1b7-2e8d373bbeda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:38.264306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ae8e8-61c5-45e3-b1b7-2e8d373bbeda service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:38.264658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ae8e8-61c5-45e3-b1b7-2e8d373bbeda service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:38.265003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ae8e8-61c5-45e3-b1b7-2e8d373bbeda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:38.314923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:38.363033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ae8e8-61c5-45e3-b1b7-2e8d373bbeda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:38.364860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ae8e8-61c5-45e3-b1b7-2e8d373bbeda service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:38.379595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1034/2070] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:39.559814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:39.572446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261e372-2c72-4f84-b4a4-54461e6d6832 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:39.584772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:39.586008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:39.586620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:39.587226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:39.604426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:39.607917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:39.631969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1037/2071] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:41.334777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:41.335363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:41.335363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:41.481515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] MFA Rules not processed for user `74fa7242b82941cc9ef9f21f122b1e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:41.487060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:41.634951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1035/2072] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:41.642402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:41.644806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:41.859516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963a3cf-5654-4fe3-bf95-4e11fe28eb5c service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:41.865130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:41.866187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:41.866835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:41.867516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:41.869067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:41.872761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:41.924284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1038/2073] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:41 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:41.928648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:41.942486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fae01d-b2f2-4f99-883c-350fa205f8ec None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.067319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.068157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.068541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.068842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.069948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.073610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.115787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1036/2074] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:41 2022] POST /identity/v3/users => generated 299 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:06:42.120647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.143300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f5269f-701c-4e61-9f4b-8c973f010df0 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.157415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.158481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.158481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.158798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.159216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.163435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.178580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1039/2075] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:42.193612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.207852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1845b2c7-de88-4841-87fa-0b9442cc9b7d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.214739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.216678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.217079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.217470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] PATH_INFO: `/v3/projects/3dc2dbf05b5c4b2bb2fb7ad3ddc18f40/users/6f06f4bf10754a0ca89a0ce4bc266484/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.244291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc2dbf05b5c4b2bb2fb7ad3ddc18f40, user_id=6f06f4bf10754a0ca89a0ce4bc266484, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.249021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.268922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1037/2076] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:42 2022] PUT /identity/v3/projects/3dc2dbf05b5c4b2bb2fb7ad3ddc18f40/users/6f06f4bf10754a0ca89a0ce4bc266484/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:42.273319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.284465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbebdf1-7e1e-4a2e-84d8-c3fc6b0e76cb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.392896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.395446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.396296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.399586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.402662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.406523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.437514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1040/2077] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:42.438418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:42.453907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d37f2-1738-4d2c-ab5b-4bd7fddceb86 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:42.460353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:42.461360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:42.461881 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:42.462361 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] PATH_INFO: `/v3/projects/3dc2dbf05b5c4b2bb2fb7ad3ddc18f40/users/6f06f4bf10754a0ca89a0ce4bc266484/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:42.466858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc2dbf05b5c4b2bb2fb7ad3ddc18f40, user_id=6f06f4bf10754a0ca89a0ce4bc266484, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:42.473711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:42.493676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1038/2078] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:42 2022] PUT /identity/v3/projects/3dc2dbf05b5c4b2bb2fb7ad3ddc18f40/users/6f06f4bf10754a0ca89a0ce4bc266484/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:44.135200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:44.142691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:44.165840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2317b2-50c2-43f2-a266-a78e23454328 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:44.230352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:44.265037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:44.265873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:44.266235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:44.266618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] PATH_INFO: `/v3/users/40e44bc9e04544be933163aae190a754` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:44.267988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e44bc9e04544be933163aae190a754)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:44.270221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:44.380465 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e44bc9e04544be933163aae190a754'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.394956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e44bc9e04544be933163aae190a754'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.412234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e44bc9e04544be933163aae190a754'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.455447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e44bc9e04544be933163aae190a754'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.458027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e44bc9e04544be933163aae190a754'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.473325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1041/2079] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:44 2022] DELETE /identity/v3/users/40e44bc9e04544be933163aae190a754 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:44.697894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:44.701231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:44.727203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a11ca-c11d-4b30-b25d-851376a34453 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:44.750315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:44.867018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:44.868382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:44.868890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:44.869410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] PATH_INFO: `/v3/users/b6e438a0e7064be4aa3e19ccbe595cb7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:44.872835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e438a0e7064be4aa3e19ccbe595cb7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:44.875283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:44.966595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e438a0e7064be4aa3e19ccbe595cb7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.969552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e438a0e7064be4aa3e19ccbe595cb7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:44.986545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:44.989503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:44.990320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e438a0e7064be4aa3e19ccbe595cb7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.016817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80053e-1d8a-4374-82f8-56aeb595802c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:45.031573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:45.035714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:45.036393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:45.036842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] PATH_INFO: `/v3/projects/08035111d1104ac8a68e36845fab2fed` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:45.038792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08035111d1104ac8a68e36845fab2fed)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:45.042213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:45.047389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e438a0e7064be4aa3e19ccbe595cb7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.073354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-780cb286-2e9d-4ffa-a933-c51d2ad9f6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e438a0e7064be4aa3e19ccbe595cb7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.088365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1039/2080] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:44 2022] DELETE /identity/v3/users/b6e438a0e7064be4aa3e19ccbe595cb7 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:45.097636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:45.098403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:45.099223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:45.131433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:45.132712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:45.163277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08035111d1104ac8a68e36845fab2fed', 'request_id': 'req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.182983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08035111d1104ac8a68e36845fab2fed', 'request_id': 'req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.197860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08035111d1104ac8a68e36845fab2fed', 'request_id': 'req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.238860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] The token cache is being invalidate because project 08035111d1104ac8a68e36845fab2fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:45.239836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.243438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.248217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1042/2081] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:44 2022] DELETE /identity/v3/projects/08035111d1104ac8a68e36845fab2fed => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:45.253374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:45.256485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:45.263770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1040/2082] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:45.271959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:45.273917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:45.319199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a66fc0-b340-4dc4-a114-5e732ae4d5f4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:45.339781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:45.341506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:45.341934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:45.342327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] PATH_INFO: `/v3/users/e0acdf96782b45d39a4cc1c20188ac9b` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:45.345885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0acdf96782b45d39a4cc1c20188ac9b)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:45.350263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:45.354161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:45.428464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41ea7f-b4d2-4b5c-afb3-a6c4cb4be82e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:45.442471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:45.453391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:45.454860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:45.457786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:45.458058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:45.464509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0acdf96782b45d39a4cc1c20188ac9b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.483996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:45.486257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0acdf96782b45d39a4cc1c20188ac9b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.488063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:45.502985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0acdf96782b45d39a4cc1c20188ac9b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.524100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1041/2083] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:45.532684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0acdf96782b45d39a4cc1c20188ac9b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.533468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-cfa0f890-1f28-4ef8-a46a-991fe5814233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0acdf96782b45d39a4cc1c20188ac9b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:45.540691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1043/2084] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:45 2022] DELETE /identity/v3/users/e0acdf96782b45d39a4cc1c20188ac9b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:46.331736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:46.347451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d45b35-b3ce-4899-bc3d-50471bb1c039 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:46.490602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:46.491442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:46.491815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:46.492229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] PATH_INFO: `/v3/projects/d09e74b746e34fd68314219faad6b428` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:46.493695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d09e74b746e34fd68314219faad6b428)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:46.495906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:46.625418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd09e74b746e34fd68314219faad6b428', 'request_id': 'req-6629b905-8a7a-4204-a24b-c4eed16f2991'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:46.636622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd09e74b746e34fd68314219faad6b428', 'request_id': 'req-6629b905-8a7a-4204-a24b-c4eed16f2991'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:46.649198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd09e74b746e34fd68314219faad6b428', 'request_id': 'req-6629b905-8a7a-4204-a24b-c4eed16f2991'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:46.667486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] The token cache is being invalidate because project d09e74b746e34fd68314219faad6b428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:46.668683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:46.673306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:46.676397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1042/2085] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:46 2022] DELETE /identity/v3/projects/d09e74b746e34fd68314219faad6b428 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:46.682544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2e16b-25ca-4a56-9205-837577ff26bf None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:46.683069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2e16b-25ca-4a56-9205-837577ff26bf None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:46.683661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b2e16b-25ca-4a56-9205-837577ff26bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:46.711834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c0b2e16b-25ca-4a56-9205-837577ff26bf None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:46.713138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c0b2e16b-25ca-4a56-9205-837577ff26bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:46.832066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:46.833636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:46.904856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1044/2086] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:46.913142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:46.913442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:46.913843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:46.946067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:46.947374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:46.966133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1045/2087] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:46.967373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:46.982774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629b905-8a7a-4204-a24b-c4eed16f2991 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:46.984261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e564c-302d-4fba-94a9-82c54633ee39 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:46.989538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:46.990546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:46.990846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:46.991126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] PATH_INFO: `/v3/projects/a3498c2154f24e2f9bcca4da53e2f705` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:46.995705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:46.996757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:46.997322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:46.998421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:46.998611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3498c2154f24e2f9bcca4da53e2f705)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:46.998786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:46.998950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.000753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.001487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.016513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:47.040677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1046/2088] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:47.043063 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.072614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394b552-f1dd-475f-82c0-3703a5adfd63 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.081380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.082800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.083286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.083870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.085457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.087876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.114107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3498c2154f24e2f9bcca4da53e2f705', 'request_id': 'req-925e0659-b75b-4bbb-b0f1-6520c68223f3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.128548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3498c2154f24e2f9bcca4da53e2f705', 'request_id': 'req-925e0659-b75b-4bbb-b0f1-6520c68223f3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.142282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3498c2154f24e2f9bcca4da53e2f705', 'request_id': 'req-925e0659-b75b-4bbb-b0f1-6520c68223f3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.147262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1047/2089] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:47 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:47.149664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.173176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] The token cache is being invalidate because project a3498c2154f24e2f9bcca4da53e2f705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:47.173549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.175647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.189706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1043/2090] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:46 2022] DELETE /identity/v3/projects/a3498c2154f24e2f9bcca4da53e2f705 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:47.194887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.203292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:47.307884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f641379-59b4-4de0-a83b-829899909a78 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.320706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.321525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.321907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.322377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.323309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.325500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.421996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1048/2091] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:47 2022] POST /identity/v3/users => generated 291 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:47.423386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.429857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:47.460389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:47.497328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebcf93-baf8-4ede-92b2-0c0201c5ddbd admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.503921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925e0659-b75b-4bbb-b0f1-6520c68223f3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.525081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.525918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.526304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.528536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.529382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.529716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.531282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.533822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.534261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.534994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] PATH_INFO: `/v3/users/afce61dcfabf460c9c8e31bd6f9f0041` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.537752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afce61dcfabf460c9c8e31bd6f9f0041)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.542877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.558382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1049/2092] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:47.581259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.608865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ab272-af5b-41a4-8321-9f1a9e54e6ad admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.626766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.629021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.629440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.629818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.632038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.635859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.670984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1050/2093] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:47.677228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:47.690096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afce61dcfabf460c9c8e31bd6f9f0041'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.691672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afce61dcfabf460c9c8e31bd6f9f0041'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.702487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0764f8-9789-460a-930a-3539d44cb1e9 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:47.708091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afce61dcfabf460c9c8e31bd6f9f0041'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.723756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:47.726686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.728156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.730091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] PATH_INFO: `/v3/projects/974add107bb04fa8990b83f5af185ad3/users/1c00b4bf1bcc49d09d8aadadc231f996/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.743960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afce61dcfabf460c9c8e31bd6f9f0041'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.759654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f23ea749-a6cf-4203-8874-119de0dbcf07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afce61dcfabf460c9c8e31bd6f9f0041'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:47.761518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=974add107bb04fa8990b83f5af185ad3, user_id=1c00b4bf1bcc49d09d8aadadc231f996, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:47.770626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1044/2094] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:47 2022] DELETE /identity/v3/users/afce61dcfabf460c9c8e31bd6f9f0041 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:47.775884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:47.805432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1051/2095] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:47 2022] PUT /identity/v3/projects/974add107bb04fa8990b83f5af185ad3/users/1c00b4bf1bcc49d09d8aadadc231f996/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:47.818796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:47.819746 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:47.820160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:47.863156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:47.870938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] MFA Rules not processed for user `1c00b4bf1bcc49d09d8aadadc231f996`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:47.874030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:48.039737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1045/2096] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:48.044987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.047056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:48.251645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24fbd8-9a3e-4dd7-b3ce-7f7907749590 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:48.268872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:48.269515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.269829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.270109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.270853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:48.272818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:48.334942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1052/2097] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:48 2022] POST /identity/v3/projects => generated 383 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:48.338920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.359561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da021-9839-4c7c-bdf9-3f9f073f1fd5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:48.544516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:48.546435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.546435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.546435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.547212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:48.549590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:48.610585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1046/2098] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:48 2022] POST /identity/v3/users => generated 298 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:48.614986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.634174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c06037-890c-4ff8-ad58-890f5d7bdc7d admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:48.646391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:48.646391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.646391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.646988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.647584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:48.649504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:48.681712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1053/2099] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:48.686852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.702657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a0366f-db04-40a0-8877-f695cb9ad2d5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:48.718074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:48.719265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.719548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.719823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] PATH_INFO: `/v3/projects/d122a8e0be0e44b08a3b2f35f903aade/users/c57478a658f9467eb07dbcf67406ac1f/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.758307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d122a8e0be0e44b08a3b2f35f903aade, user_id=c57478a658f9467eb07dbcf67406ac1f, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:48.761151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:48.792686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1047/2100] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:48 2022] PUT /identity/v3/projects/d122a8e0be0e44b08a3b2f35f903aade/users/c57478a658f9467eb07dbcf67406ac1f/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:48.796867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.815176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a751c4e-fdd9-4b8f-b84b-1869de478e7a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:48.879042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:48.881097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:48.985297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:48.985920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:48.986203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:48.986478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:48.987137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:48.992964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:49.015002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1054/2101] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:06:49.022980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:49.040734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4a568-debc-4c35-8865-480a6c934206 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:49.053699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf7818-fcdb-4fbb-a0ed-d2f1f0d2c800 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:49.054347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0725-2dd1-475d-9b85-b86613b4f64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:49.055141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0725-2dd1-475d-9b85-b86613b4f64e admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:49.055629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0725-2dd1-475d-9b85-b86613b4f64e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:49.056362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0725-2dd1-475d-9b85-b86613b4f64e admin admin] PATH_INFO: `/v3/projects/d122a8e0be0e44b08a3b2f35f903aade/users/c57478a658f9467eb07dbcf67406ac1f/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:49.062446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca0725-2dd1-475d-9b85-b86613b4f64e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d122a8e0be0e44b08a3b2f35f903aade, user_id=c57478a658f9467eb07dbcf67406ac1f, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:49.071407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca0725-2dd1-475d-9b85-b86613b4f64e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:49.075055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:49.076071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:49.076649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:49.077088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] PATH_INFO: `/v3/projects/bd6291120b6740d48f087f05eabbae16` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:49.079282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd6291120b6740d48f087f05eabbae16)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:49.081621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:49.098300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1055/2102] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:49 2022] PUT /identity/v3/projects/d122a8e0be0e44b08a3b2f35f903aade/users/c57478a658f9467eb07dbcf67406ac1f/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:49.111614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:49.111930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:49.112244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:49.113975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:49.169276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:49.187623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] MFA Rules not processed for user `c57478a658f9467eb07dbcf67406ac1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:49.191415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:49.217144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd6291120b6740d48f087f05eabbae16', 'request_id': 'req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.229974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd6291120b6740d48f087f05eabbae16', 'request_id': 'req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.241841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd6291120b6740d48f087f05eabbae16', 'request_id': 'req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.258989 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] The token cache is being invalidate because project bd6291120b6740d48f087f05eabbae16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:49.260306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.261428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.263584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1048/2103] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:48 2022] DELETE /identity/v3/projects/bd6291120b6740d48f087f05eabbae16 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:49.267721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:49.270123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:49.412035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b00e32-f6a1-47a5-92c1-cf67ef02a688 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:49.416471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:49.417128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:49.417415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:49.417845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] PATH_INFO: `/v3/users/355a960670e140dab1c17e7e6e33c48f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:49.419935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=355a960670e140dab1c17e7e6e33c48f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:49.422379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:49.459903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1056/2104] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:06:49.484069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:49.489351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:49.531185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '355a960670e140dab1c17e7e6e33c48f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.533984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '355a960670e140dab1c17e7e6e33c48f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.555098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '355a960670e140dab1c17e7e6e33c48f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.574190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '355a960670e140dab1c17e7e6e33c48f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.584634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '355a960670e140dab1c17e7e6e33c48f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:49.598046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1049/2105] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:49 2022] DELETE /identity/v3/users/355a960670e140dab1c17e7e6e33c48f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:49.656108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719a14c-a5a2-4ae2-a300-386887c0cfe6 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:49.772317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:49.773288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:49.773881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:49.774312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:49.788190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:49.966799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:49.968704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884a1970-d329-4043-8647-c7f65c6be268 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:49.980531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1057/2106] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:50.415448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:50.418574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:50.461475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:50.577224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e26ae-051a-4cb7-9cd8-20a88d25ddfa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:50.587783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:50.588709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:50.588998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:50.590455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] PATH_INFO: `/v3/projects/f8d3fd75cd5c4f04ad4a10de6e0f3f9f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:50.591466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d3fd75cd5c4f04ad4a10de6e0f3f9f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:50.593523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:50.687007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d3fd75cd5c4f04ad4a10de6e0f3f9f', 'request_id': 'req-0ab90103-9f06-45ed-95ad-463a91cc271b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:50.702369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d3fd75cd5c4f04ad4a10de6e0f3f9f', 'request_id': 'req-0ab90103-9f06-45ed-95ad-463a91cc271b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:50.717338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d3fd75cd5c4f04ad4a10de6e0f3f9f', 'request_id': 'req-0ab90103-9f06-45ed-95ad-463a91cc271b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:50.741104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] The token cache is being invalidate because project f8d3fd75cd5c4f04ad4a10de6e0f3f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:50.742020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:50.742982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0ab90103-9f06-45ed-95ad-463a91cc271b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:50.751025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1050/2107] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:50 2022] DELETE /identity/v3/projects/f8d3fd75cd5c4f04ad4a10de6e0f3f9f => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:50.755631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:50.755917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:50.756216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:50.809468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:50.810771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:50.830316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:51.017707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1058/2108] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:51.025740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.026173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.026555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.069973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:51.073769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:51.084960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1051/2109] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:51.089834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:51.109563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34559ef3-350e-4dd7-8c78-ef9a3c459823 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:51.131786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:51.132927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.133783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.134302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.135375 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:51.135727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:51.142338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:51.179435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1059/2110] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:51.181190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:51.202532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b7af0-083a-4b30-9c2b-8bec69cb3c46 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:51.206485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:51.210584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.210584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.210584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.210584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:51.211693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:51.295533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1052/2111] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:51 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:51.299723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:51.331770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09d689-9089-4f52-b8a8-7408b6797a9b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:51.572055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:51.575243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.575791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.576533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.577464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:51.580058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:51.649916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1060/2112] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:51 2022] POST /identity/v3/users => generated 293 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:51.651639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:51.668385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe4cb4-d99e-4dd6-9f68-29d3abfdd2e5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:51.677187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:51.678209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.678714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.680890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.681901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:51.693364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:51.718449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1053/2113] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:51.718888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:51.743933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93cc7f4-653d-4d25-9d99-3d864e01da08 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:51.749669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e8f15-0e04-43b0-a532-7c89483e00ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:51.750528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e8f15-0e04-43b0-a532-7c89483e00ea admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.750718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e8f15-0e04-43b0-a532-7c89483e00ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.751082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e8f15-0e04-43b0-a532-7c89483e00ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.751810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e8f15-0e04-43b0-a532-7c89483e00ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:51.753536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e8f15-0e04-43b0-a532-7c89483e00ea admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:51.768734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1061/2114] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:51.772663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:51.801987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec914a0-59fe-4160-9ebe-a796b143125f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:51.809566 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:51.810424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.810706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.810984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] PATH_INFO: `/v3/projects/ca3032cfac654ef1aa57cac5306e4120/users/c23f68ed78734e988613fe418762818a/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.833233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca3032cfac654ef1aa57cac5306e4120, user_id=c23f68ed78734e988613fe418762818a, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:51.841940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:51.884192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1054/2115] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:51 2022] PUT /identity/v3/projects/ca3032cfac654ef1aa57cac5306e4120/users/c23f68ed78734e988613fe418762818a/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:51.884967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:51.885458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:51.885996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:51.938158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] MFA Rules not processed for user `c23f68ed78734e988613fe418762818a`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:51.940188 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:52.130055 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1062/2116] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:52.130903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:52.278539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:52.314929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eca05a-20fc-4410-acb2-9332c321006e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:52.328481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:52.330014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:52.330866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:52.331940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:52.334059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:52.337985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:52.434299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:52.437257 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1055/2117] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:52.442854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:52.454736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259e6ac-1243-47e2-a008-382d125148a5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:52.689956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:52.690703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:52.690848 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:52.691123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:52.691774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:52.693825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:52.785874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1063/2118] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:52 2022] POST /identity/v3/users => generated 292 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:52.786689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:52.805242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bcd73-27a9-49d1-8446-67519edc21db admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:52.848354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:52.854194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:52.856452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:52.856962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:52.861585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:52.863366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:52.893648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1056/2119] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:52.897744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:52.912319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e2f5c7-7823-4fd4-86e9-16ec49832d58 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:52.940992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27aeb57-f0d1-4282-a5dd-6ac8b65e6bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:52.941627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27aeb57-f0d1-4282-a5dd-6ac8b65e6bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:52.941903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27aeb57-f0d1-4282-a5dd-6ac8b65e6bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:52.942171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27aeb57-f0d1-4282-a5dd-6ac8b65e6bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:52.942845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27aeb57-f0d1-4282-a5dd-6ac8b65e6bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:52.944512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27aeb57-f0d1-4282-a5dd-6ac8b65e6bc6 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:52.985242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1064/2120] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:52.985403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:52.996829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8938038-795f-43b9-8e53-54130e9976b4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:53.004922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189887da-09de-4dc9-92a5-b17a0128d869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:53.005620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189887da-09de-4dc9-92a5-b17a0128d869 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.005905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189887da-09de-4dc9-92a5-b17a0128d869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.006183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189887da-09de-4dc9-92a5-b17a0128d869 admin admin] PATH_INFO: `/v3/projects/21c4eb2e281542818c0f97c3082e7d09/users/1dd2f49ab32f40ba81d284f9944f784d/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.052637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189887da-09de-4dc9-92a5-b17a0128d869 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c4eb2e281542818c0f97c3082e7d09, user_id=1dd2f49ab32f40ba81d284f9944f784d, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:53.057363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189887da-09de-4dc9-92a5-b17a0128d869 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:53.095686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1057/2121] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:52 2022] PUT /identity/v3/projects/21c4eb2e281542818c0f97c3082e7d09/users/1dd2f49ab32f40ba81d284f9944f784d/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:53.108729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.109019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.109427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.166759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] MFA Rules not processed for user `1dd2f49ab32f40ba81d284f9944f784d`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:53.168498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:53.302156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.302735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.302978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.314408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1065/2122] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:53.319184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:53.368366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] MFA Rules not processed for user `6f06f4bf10754a0ca89a0ce4bc266484`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:53.370087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:53.453096 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99100e5a-528e-4f4d-a46c-7083f20a8642 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:53.461398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:53.462387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.462737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.463078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.463822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:53.466888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:53.505774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:53.520162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1066/2123] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:53 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:53.521291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:53.560998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1058/2124] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:53.587030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:53.588754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:53.610230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35f4e11-71f8-4e08-abb0-9d46fccaf8e8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:53.655641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde4daf-01be-4524-be46-de9484d2e80e admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:53.664043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:53.666442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.667238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.668630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.669434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:53.673157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:53.721944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1067/2125] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:53 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:53.723288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:53.725918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:53.728744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:53.729788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.730595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.731458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] PATH_INFO: `/v3/users/0517d50b405e4bce991bb22eef6076a4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.734146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0517d50b405e4bce991bb22eef6076a4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:53.738092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:53.770379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:53.823575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0517d50b405e4bce991bb22eef6076a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:53.825873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0517d50b405e4bce991bb22eef6076a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:53.837294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0517d50b405e4bce991bb22eef6076a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:53.854005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891bb01-777a-476b-a79f-bf10dd411034 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:53.854224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0517d50b405e4bce991bb22eef6076a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:53.858895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:53.859615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.859907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.860230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.862319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:53.865213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0517d50b405e4bce991bb22eef6076a4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:53.874416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1059/2126] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:06:53 2022] DELETE /identity/v3/users/0517d50b405e4bce991bb22eef6076a4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:53.876793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:53.878690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:53.963593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:53.964272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:53.975847 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f2e70-f237-4a26-acdb-edf53ef01437 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:53.976479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1068/2127] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:53.983693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:53.983995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:53.984305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:53.985393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:53.985393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:53.986864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:54.008633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:54.012173 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1060/2128] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:54.023554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff4dbf0-ea68-44b9-9096-6c3b3686c459 service glance] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:54.030215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:54.030937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.031366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.031752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] PATH_INFO: `/v3/projects/7f5ce9337828430fbe39fa565bbf5399/users/e9fb0226c99e412cb2b50d09ff9b1209/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.056268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5ce9337828430fbe39fa565bbf5399, user_id=e9fb0226c99e412cb2b50d09ff9b1209, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:54.060490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:54.076854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:54.087056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1069/2129] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:54 2022] PUT /identity/v3/projects/7f5ce9337828430fbe39fa565bbf5399/users/e9fb0226c99e412cb2b50d09ff9b1209/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:54.091184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:54.109806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14aa54e-54fc-4f8f-9781-75c0af471b50 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:54.175179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:54.177079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:54.251861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:54.252806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.255573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.256031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.257022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:54.259644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:54.277744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1061/2130] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:54 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:54.280640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:54.296211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedf09f-aac2-4eb8-9f09-3f19f6caf122 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:54.314467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e96ec-ffe6-4a6c-9772-6134dced1fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:54.315729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e96ec-ffe6-4a6c-9772-6134dced1fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.316962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e96ec-ffe6-4a6c-9772-6134dced1fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.317921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e96ec-ffe6-4a6c-9772-6134dced1fd7 admin admin] PATH_INFO: `/v3/projects/7f5ce9337828430fbe39fa565bbf5399/users/e9fb0226c99e412cb2b50d09ff9b1209/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.325546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e96ec-ffe6-4a6c-9772-6134dced1fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5ce9337828430fbe39fa565bbf5399, user_id=e9fb0226c99e412cb2b50d09ff9b1209, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:54.328724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e96ec-ffe6-4a6c-9772-6134dced1fd7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:54.346074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1062/2131] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:54 2022] PUT /identity/v3/projects/7f5ce9337828430fbe39fa565bbf5399/users/e9fb0226c99e412cb2b50d09ff9b1209/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:54.347741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bc95e-6e8c-4721-9d2c-508fbe8c94ed None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.348061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bc95e-6e8c-4721-9d2c-508fbe8c94ed None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.348341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bc95e-6e8c-4721-9d2c-508fbe8c94ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.381683 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6b5bc95e-6e8c-4721-9d2c-508fbe8c94ed None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:54.382924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6b5bc95e-6e8c-4721-9d2c-508fbe8c94ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:54.403001 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:54.450127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1676164-32a3-467e-a4c5-ce9b80565983 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:54.493386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1063/2132] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:54.496825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.498876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.499909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.597111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] MFA Rules not processed for user `e9fb0226c99e412cb2b50d09ff9b1209`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:54.600693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:54.681301 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:54.682091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.682449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.682728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] PATH_INFO: `/v3/projects/0ee7c5a0b37b4192a3746ee760ecefa5` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.689336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee7c5a0b37b4192a3746ee760ecefa5)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:54.695163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:54.798113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1064/2133] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:54.800300 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:54.804586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:54.824574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee7c5a0b37b4192a3746ee760ecefa5', 'request_id': 'req-d159c8d8-7703-498f-aa1f-633ea6a99164'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:54.831956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee7c5a0b37b4192a3746ee760ecefa5', 'request_id': 'req-d159c8d8-7703-498f-aa1f-633ea6a99164'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:54.844634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee7c5a0b37b4192a3746ee760ecefa5', 'request_id': 'req-d159c8d8-7703-498f-aa1f-633ea6a99164'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:54.867789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] The token cache is being invalidate because project 0ee7c5a0b37b4192a3746ee760ecefa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:06:54.868614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:54.868887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:06:54.871938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1070/2134] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:06:54 2022] DELETE /identity/v3/projects/0ee7c5a0b37b4192a3746ee760ecefa5 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:06:54.873496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:54.874994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:54.920101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9c1fb-1905-438b-a2b6-b143c3f9076c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:54.931052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:54.962414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d159c8d8-7703-498f-aa1f-633ea6a99164 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:54.970531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:54.971266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:54.971626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:54.971983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:54.974405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:55.005315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:55.022898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:55.023902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:55.024618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:55.025071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:55.028339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:55.081900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:55.083659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e37d35-aa15-4503-841b-668f46b5df37 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:55.098147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1071/2135] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:55.100411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:55.100846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:55.101127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:55.171777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:55.173656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:55.175369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:55.177009 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c29dd8-056f-44a5-96d9-d925e748f803 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:55.209235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1065/2136] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:06:55.408069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1072/2137] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:55.413377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:55.413790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:55.414019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:55.469519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:55.471125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:55.492208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1066/2138] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:55.493687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:55.513959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c726c-c1f6-48e8-9db7-7b1acf0cdff4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:55.532469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:55.533104 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:55.533385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:55.533656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:55.534316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:06:55.534628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:55.536278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:55.598421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1073/2139] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:06:55.602699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:55.616079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971939ca-9670-4a0f-8673-fec7da800457 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:55.625969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:55.626861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:55.627145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:55.627461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:55.629598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:55.632843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:55.698502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1067/2140] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:06:55 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:55.703155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:55.762195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0014def7-0f0d-4e4a-abd0-55ef50e3f81a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:55.948156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:55.951751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:55.962049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:55.962954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:55.963396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:55.963884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:55.965447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:55.967627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:56.019600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1074/2141] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:06:55 2022] POST /identity/v3/users => generated 289 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:06:56.029148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:56.049211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c327bc72-f1fc-4f1e-aa85-ba50363e73d6 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:56.076329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:56.076901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:56.077332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:56.077463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:56.078134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:56.082442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:56.082442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:56.088555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a8d22-de7b-420a-be71-9afa86cea1d2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:56.104161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:56.104641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:56.105440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:56.106039 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:56.109599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:56.122637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1075/2142] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:56.139527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:56.176430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35ecfdf-e6ff-448a-bf41-d5be705d00f2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:56.186393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:56.187805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:56.188410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:56.188995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:56.190537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:56.192646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:56.217195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1076/2143] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:06:56.221597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:56.237771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:56.240104 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83a842-5e3c-4d66-9a24-e4de349b783d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:56.253120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c1bce5-192c-4be9-8b17-5c88124bdea5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:56.266710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1068/2144] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:56.276527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:56.280694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:56.281016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:56.281573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] PATH_INFO: `/v3/projects/00b397afc10243bc9b93548873add6ef/users/81d937b8a6c842819458b6e3c23f80fa/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:56.320430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00b397afc10243bc9b93548873add6ef, user_id=81d937b8a6c842819458b6e3c23f80fa, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:56.330306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:56.361435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1077/2145] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:06:56 2022] PUT /identity/v3/projects/00b397afc10243bc9b93548873add6ef/users/81d937b8a6c842819458b6e3c23f80fa/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:06:56.369122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d781-5b68-4ee8-9003-81de483dca8d None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:56.369444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d781-5b68-4ee8-9003-81de483dca8d None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:56.369690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d781-5b68-4ee8-9003-81de483dca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:56.444761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9893d781-5b68-4ee8-9003-81de483dca8d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:56.450139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9893d781-5b68-4ee8-9003-81de483dca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:56.477925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:06:56.638736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1069/2146] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:56.646631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:56.648853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:56.802036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ff840-6bf7-4f0c-8217-3e82fd05a127 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:56.828200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:56.828835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:56.829119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:56.829391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:56.845157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:56.847117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:56.859941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1078/2147] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:06:57.199588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:57.200223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:57.200223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:57.266117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] MFA Rules not processed for user `fe027aad062342688c587428f2bfbfe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:06:57.267446 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:57.297546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:57.448065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1070/2148] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:06:57.465868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa4b1a-b9b6-41b5-916f-f201bc4b09ff service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:57.476660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:57.477290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:57.477576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:57.477905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:57.482783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:57.714605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:57.716166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:57.736780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1079/2149] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:06:59.407611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:06:59.434365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646f72dd-f66e-4723-9c84-f8aac48d062f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:06:59.441199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:06:59.446224 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:06:59.446682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:06:59.447065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:06:59.455580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:06:59.644272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:06:59.648752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1cb17c-fd93-42ff-bd1f-47ecd66e728d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:06:59.677584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1071/2150] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:07:02.110105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:02.112436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:02.142296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e312412-640d-488c-a04d-fb3bf7fef07b service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:02.173446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:02.178548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:02.181153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:02.181404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] PATH_INFO: `/v3/users/1c00b4bf1bcc49d09d8aadadc231f996` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:02.185011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c00b4bf1bcc49d09d8aadadc231f996)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:02.197569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:02.289119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:02.445156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c00b4bf1bcc49d09d8aadadc231f996'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:02.459743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c00b4bf1bcc49d09d8aadadc231f996'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:02.472729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c00b4bf1bcc49d09d8aadadc231f996'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:02.508125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c00b4bf1bcc49d09d8aadadc231f996'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:02.516513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c00b4bf1bcc49d09d8aadadc231f996'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:02.524878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1080/2151] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:07:02 2022] DELETE /identity/v3/users/1c00b4bf1bcc49d09d8aadadc231f996 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:02.530410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:02.531522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:02.533612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:02.570701 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:02.575182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:02.738904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:02.766324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1072/2152] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:07:02.785483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:02.785838 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:02.952075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185add16-ecf0-41e0-acda-7a189bcbc46c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:02.962057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:02.962696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:02.962982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:02.963261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:02.989804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:02.991284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:03.035107 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1081/2153] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:07:04.112700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:04.115153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:04.143828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ebaf6-0627-43d4-888d-e18e26b42a16 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:04.154181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:04.155377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:04.155946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:04.156502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] PATH_INFO: `/v3/projects/974add107bb04fa8990b83f5af185ad3` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:04.158575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=974add107bb04fa8990b83f5af185ad3)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:04.160974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:04.304770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '974add107bb04fa8990b83f5af185ad3', 'request_id': 'req-4c862088-b7e2-4fa9-8f84-fd0e969202dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.324627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '974add107bb04fa8990b83f5af185ad3', 'request_id': 'req-4c862088-b7e2-4fa9-8f84-fd0e969202dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.335189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '974add107bb04fa8990b83f5af185ad3', 'request_id': 'req-4c862088-b7e2-4fa9-8f84-fd0e969202dc'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.360857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] The token cache is being invalidate because project 974add107bb04fa8990b83f5af185ad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:07:04.361148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.365839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.369014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1073/2154] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:07:04 2022] DELETE /identity/v3/projects/974add107bb04fa8990b83f5af185ad3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:04.374010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:04.387579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e049c-ba0c-45fe-8a76-dbb1c7261ccc service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:04.517642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:04.518299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:04.519098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:04.519388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] PATH_INFO: `/v3/users/c57478a658f9467eb07dbcf67406ac1f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:04.521760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c57478a658f9467eb07dbcf67406ac1f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:04.524252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:04.647522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:04.649383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:04.652353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:04.678462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c862088-b7e2-4fa9-8f84-fd0e969202dc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:04.686108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c57478a658f9467eb07dbcf67406ac1f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.692439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:04.693593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:04.693877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:04.694152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] PATH_INFO: `/v3/users/c23f68ed78734e988613fe418762818a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:04.699315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c57478a658f9467eb07dbcf67406ac1f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.703810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c23f68ed78734e988613fe418762818a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:04.711096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:04.727518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c57478a658f9467eb07dbcf67406ac1f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.779787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c57478a658f9467eb07dbcf67406ac1f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.781499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f3b68e11-ab56-445c-9149-b31bec567160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c57478a658f9467eb07dbcf67406ac1f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.793114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1082/2155] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:07:04 2022] DELETE /identity/v3/users/c57478a658f9467eb07dbcf67406ac1f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:04.840656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c23f68ed78734e988613fe418762818a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.841367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c23f68ed78734e988613fe418762818a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.860668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c23f68ed78734e988613fe418762818a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.888912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c23f68ed78734e988613fe418762818a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.910750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c23f68ed78734e988613fe418762818a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:04.947171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1074/2156] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:07:04 2022] DELETE /identity/v3/users/c23f68ed78734e988613fe418762818a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:04.952724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:04.953024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:04.953303 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:04.991704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:05.001655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:05.219432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1083/2157] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:05.247910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:05.249404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:05.411537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:05.422651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c2b8d3-12ae-4c8f-bcf2-bfa07c453c50 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:05.434636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f0e6fa-9da5-4abf-83bb-10169ca55f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:05.435272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e6fa-9da5-4abf-83bb-10169ca55f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:05.435560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e6fa-9da5-4abf-83bb-10169ca55f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:05.435827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f0e6fa-9da5-4abf-83bb-10169ca55f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:05.476669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0e6fa-9da5-4abf-83bb-10169ca55f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:05.478548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f0e6fa-9da5-4abf-83bb-10169ca55f6c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:05.522895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1075/2158] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:07:06.570102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:06.571818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:06.601414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757f0bb7-75b0-4156-9175-8c4976ca21e6 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:06.616913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:06.618127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:06.618527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:06.618816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] PATH_INFO: `/v3/projects/d122a8e0be0e44b08a3b2f35f903aade` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:06.624983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d122a8e0be0e44b08a3b2f35f903aade)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:06.632635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:06.762661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd122a8e0be0e44b08a3b2f35f903aade', 'request_id': 'req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:06.776908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd122a8e0be0e44b08a3b2f35f903aade', 'request_id': 'req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:06.786159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd122a8e0be0e44b08a3b2f35f903aade', 'request_id': 'req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:06.817437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] The token cache is being invalidate because project d122a8e0be0e44b08a3b2f35f903aade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:07:06.817736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:06.818429 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:06.821267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1084/2159] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:07:06 2022] DELETE /identity/v3/projects/d122a8e0be0e44b08a3b2f35f903aade => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:06.832458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a4743-78cd-4ec3-b791-7e00641fe11b None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:06.833430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a4743-78cd-4ec3-b791-7e00641fe11b None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:06.834006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a4743-78cd-4ec3-b791-7e00641fe11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:06.883499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-317a4743-78cd-4ec3-b791-7e00641fe11b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:06.885619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-317a4743-78cd-4ec3-b791-7e00641fe11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:07.029900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.031421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:07.117056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1076/2160] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:07.123267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.123565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.124676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.198266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:07.224252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:07.228546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:07.260757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1077/2161] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:07.263145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.274411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1f9fe-a971-4055-b5ee-79a33b2cf4ad admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.286754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42a97e-583b-421a-a621-69ad5fd32572 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.287858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.289048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.290050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.290998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] PATH_INFO: `/v3/projects/ca3032cfac654ef1aa57cac5306e4120` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.293508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca3032cfac654ef1aa57cac5306e4120)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.296308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.302444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.304232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.304735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.305109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.306126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:07:07.309125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.311136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.339438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1078/2162] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:07:07.347156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.397024 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4aa520-8db0-468f-a4b3-91ffd2c35690 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.419120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.423737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.424850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.425784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.427970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.434510 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.475004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca3032cfac654ef1aa57cac5306e4120', 'request_id': 'req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.492651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca3032cfac654ef1aa57cac5306e4120', 'request_id': 'req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.532648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca3032cfac654ef1aa57cac5306e4120', 'request_id': 'req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.547802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1079/2163] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:07:07 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:07:07.555416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.566740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] The token cache is being invalidate because project ca3032cfac654ef1aa57cac5306e4120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:07:07.567654 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.571187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.577047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1085/2164] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:07:07 2022] DELETE /identity/v3/projects/ca3032cfac654ef1aa57cac5306e4120 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:07:07.581058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.582839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:07.710672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dfca9d-1ab4-41b6-b109-1f0590e3a1fc admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.747956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd808f-1241-47a9-ad1d-ca579e527e7b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.761210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.765251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.765694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.766083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.767023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.776997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.813331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.814092 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.814378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.814910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] PATH_INFO: `/v3/users/1dd2f49ab32f40ba81d284f9944f784d` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.816527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd2f49ab32f40ba81d284f9944f784d)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.823869 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.824916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1080/2165] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:07:07 2022] POST /identity/v3/users => generated 286 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:07:07.835225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.838099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:07.865586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:07.873918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde6d3a-eaf8-4a52-b602-28280624a507 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.879775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.880457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.880745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.881021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.882040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.885831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.918148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1081/2166] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:07:07.922283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:07.944540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd2f49ab32f40ba81d284f9944f784d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.951438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cb245-792a-40bd-b869-ed376d45c92c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:07.960066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:07.960947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:07.961287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd2f49ab32f40ba81d284f9944f784d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:07.961568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:07.962003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:07.962870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:07.972630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:07.980868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd2f49ab32f40ba81d284f9944f784d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:08.005651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1082/2167] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:07:08.006680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd2f49ab32f40ba81d284f9944f784d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:08.008872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-a2108c5a-7186-4133-922f-a5f8dcccf682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd2f49ab32f40ba81d284f9944f784d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:08.010990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:08.028321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1086/2168] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:07:07 2022] DELETE /identity/v3/users/1dd2f49ab32f40ba81d284f9944f784d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:08.030914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:08.066404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d448c-eba3-4996-80f7-9c89fe2a49b5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:08.084337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:08.085248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:08.085684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:08.098983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:08.158600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] MFA Rules not processed for user `81d937b8a6c842819458b6e3c23f80fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:08.159755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:08.240527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247f47-256d-43ad-8e02-581d3a2c6393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:08.242293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247f47-256d-43ad-8e02-581d3a2c6393 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:08.242293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247f47-256d-43ad-8e02-581d3a2c6393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:08.242924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247f47-256d-43ad-8e02-581d3a2c6393 admin admin] PATH_INFO: `/v3/projects/48c48ea6637c49d7b7cec6e805c4be87/users/ce412ceead11449aa5969320405e9f3a/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:08.300214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247f47-256d-43ad-8e02-581d3a2c6393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48c48ea6637c49d7b7cec6e805c4be87, user_id=ce412ceead11449aa5969320405e9f3a, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:08.302424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247f47-256d-43ad-8e02-581d3a2c6393 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:08.356121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1083/2169] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:07:08 2022] PUT /identity/v3/projects/48c48ea6637c49d7b7cec6e805c4be87/users/ce412ceead11449aa5969320405e9f3a/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:08.359135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:08.359379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:08.359658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:08.388252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1087/2170] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:08.471564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] MFA Rules not processed for user `ce412ceead11449aa5969320405e9f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:08.473147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:08.592753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:08.594546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:08.622531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1084/2171] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:08.639716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:08.641512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:08.784781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f70301-ff39-4c03-82ac-14d020371576 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:08.809343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984be26-a9d4-4cfc-bc56-32e8a803af85 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:08.820649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:08.833346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:08.838539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:08.839455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:08.839455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:08.862181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:08.868807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:08.897839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1085/2172] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:07:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:07:08.931569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:08.933130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:08.933420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:08.933868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:09.115799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:09.117667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:09.147336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1088/2173] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:07:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:07:09.414985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:09.418264 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:09.547925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:09.639801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9324b15-4639-489b-9d13-9a72019fbf23 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:09.652801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:09.653500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:09.654309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:09.654637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] PATH_INFO: `/v3/projects/21c4eb2e281542818c0f97c3082e7d09` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:09.656268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c4eb2e281542818c0f97c3082e7d09)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:09.658381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:09.836306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c4eb2e281542818c0f97c3082e7d09', 'request_id': 'req-3dd79df5-7458-4918-a227-42055dab885a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:09.877454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c4eb2e281542818c0f97c3082e7d09', 'request_id': 'req-3dd79df5-7458-4918-a227-42055dab885a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:09.889014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c4eb2e281542818c0f97c3082e7d09', 'request_id': 'req-3dd79df5-7458-4918-a227-42055dab885a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:09.947522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] The token cache is being invalidate because project 21c4eb2e281542818c0f97c3082e7d09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:07:09.947849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:09.948740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3dd79df5-7458-4918-a227-42055dab885a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:09.955691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1086/2174] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:07:09 2022] DELETE /identity/v3/projects/21c4eb2e281542818c0f97c3082e7d09 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:09.956128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:10.019821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04607c-f8e6-4fca-b98c-8d333213c560 service nova] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:10.190742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:10.191432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:10.191740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:10.192054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] PATH_INFO: `/v3/users/e9fb0226c99e412cb2b50d09ff9b1209` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:10.193371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fb0226c99e412cb2b50d09ff9b1209)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:10.195387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:10.288088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fb0226c99e412cb2b50d09ff9b1209'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:10.316177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fb0226c99e412cb2b50d09ff9b1209'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:10.319829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:10.336203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fb0226c99e412cb2b50d09ff9b1209'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:10.370154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fb0226c99e412cb2b50d09ff9b1209'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:10.372124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fb0226c99e412cb2b50d09ff9b1209'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:10.385009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1089/2175] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:07:09 2022] DELETE /identity/v3/users/e9fb0226c99e412cb2b50d09ff9b1209 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:10.704391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3013a8-6cd4-4795-a0cc-73e16760b18c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:10.704770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3013a8-6cd4-4795-a0cc-73e16760b18c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:10.704952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3013a8-6cd4-4795-a0cc-73e16760b18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:10.757403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8f3013a8-6cd4-4795-a0cc-73e16760b18c None None] MFA Rules not processed for user `ce412ceead11449aa5969320405e9f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:10.758754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f3013a8-6cd4-4795-a0cc-73e16760b18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:10.942237 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:10.944843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:11.028291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1087/2176] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:11.183111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e23db-e500-4800-aab1-966e15ae5774 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:11.202004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:11.202876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:11.203316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:11.203750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] PATH_INFO: `/v3/projects/7f5ce9337828430fbe39fa565bbf5399` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:11.206379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5ce9337828430fbe39fa565bbf5399)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:11.209410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:11.358116 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5ce9337828430fbe39fa565bbf5399', 'request_id': 'req-60f5269a-d2df-47ea-b0be-bf28733ae679'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:11.378734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5ce9337828430fbe39fa565bbf5399', 'request_id': 'req-60f5269a-d2df-47ea-b0be-bf28733ae679'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:11.399918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5ce9337828430fbe39fa565bbf5399', 'request_id': 'req-60f5269a-d2df-47ea-b0be-bf28733ae679'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:11.424999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] The token cache is being invalidate because project 7f5ce9337828430fbe39fa565bbf5399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:07:11.425474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:11.426346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-60f5269a-d2df-47ea-b0be-bf28733ae679 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:07:11.429260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1090/2177] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:07:10 2022] DELETE /identity/v3/projects/7f5ce9337828430fbe39fa565bbf5399 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:11.434663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:11.434957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:11.435231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:11.472600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:11.473850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:11.627123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1088/2178] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:11.638251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:11.638564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:11.638817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:11.687913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:11.700227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:11.726513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1091/2179] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:11.726645 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:11.743417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710e59-2cde-443b-9d59-ba3fb89e9fb9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:11.753263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:11.754522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:11.755245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:11.755245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:11.756371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:07:11.756617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:11.759273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:11.766016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:11.792377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1089/2180] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:07:11.794473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:11.826196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832c56a-f4d4-4b4a-9336-dab2374df900 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:11.832118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:11.833136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:11.833699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:11.834181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:11.835051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:11.837315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:11.886865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1092/2181] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:07:11 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:07:11.891020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:11.904532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3d011-7ce5-4dc3-81df-ca96d499d707 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:12.114388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:12.115317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:12.115879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:12.116275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:12.117263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:12.122136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:12.194736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1090/2182] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:07:11 2022] POST /identity/v3/users => generated 287 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:07:12.196494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:12.211065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb36fb-dc59-4b2a-ab99-330d134ba52a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:12.251496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:12.253971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:12.254823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:12.255118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:12.255855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:12.257529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:12.274875 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1093/2183] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:07:12.275919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:12.292201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ffe9e6-7906-4e11-b579-07614f49fe06 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:12.306058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7250c6-e7fc-44d5-a4cf-167be086bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:12.306750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7250c6-e7fc-44d5-a4cf-167be086bb78 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:12.307745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7250c6-e7fc-44d5-a4cf-167be086bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:12.308041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7250c6-e7fc-44d5-a4cf-167be086bb78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:12.308898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7250c6-e7fc-44d5-a4cf-167be086bb78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:12.314753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7250c6-e7fc-44d5-a4cf-167be086bb78 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:12.340515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1091/2184] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:07:12.341321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:12.360774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d7b5ad-3791-4c21-b447-b7aedb0e6317 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:12.386290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:12.388730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:12.389051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:12.389293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] PATH_INFO: `/v3/projects/42099ef23496441fab42f876b9e39efd/users/d8c0e96bc0584d8b9be73ae920bbb055/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:12.449211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] RBAC: Authorizing `identity:create_grant(project_id=42099ef23496441fab42f876b9e39efd, user_id=d8c0e96bc0584d8b9be73ae920bbb055, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:12.453074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:12.476283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1094/2185] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:07:12 2022] PUT /identity/v3/projects/42099ef23496441fab42f876b9e39efd/users/d8c0e96bc0584d8b9be73ae920bbb055/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:12.483027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc639-27c3-4d68-a15f-def2378984ac None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:12.483027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc639-27c3-4d68-a15f-def2378984ac None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:12.483027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efc639-27c3-4d68-a15f-def2378984ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:12.514320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-79efc639-27c3-4d68-a15f-def2378984ac None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:12.516429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-79efc639-27c3-4d68-a15f-def2378984ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:12.683972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1092/2186] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:12.698582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:12.701476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:12.709047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:12.887032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6749c-a30d-41b2-986a-a647c7d2c00f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:12.894702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b82a6-0e7f-4cdf-8e12-adcceb6431ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:12.895667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b82a6-0e7f-4cdf-8e12-adcceb6431ce service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:12.896337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b82a6-0e7f-4cdf-8e12-adcceb6431ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:12.898718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b82a6-0e7f-4cdf-8e12-adcceb6431ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:12.945601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b82a6-0e7f-4cdf-8e12-adcceb6431ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:12.968650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b82a6-0e7f-4cdf-8e12-adcceb6431ce service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:12.986764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1095/2187] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:07:13.125263 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53459db-f293-457f-999b-136d44e72b52 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:13.125738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53459db-f293-457f-999b-136d44e72b52 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:13.125827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53459db-f293-457f-999b-136d44e72b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:13.195569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f53459db-f293-457f-999b-136d44e72b52 None None] MFA Rules not processed for user `81d937b8a6c842819458b6e3c23f80fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:13.200627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f53459db-f293-457f-999b-136d44e72b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:13.410451 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1093/2188] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:20.565732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:20.566926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:20.567583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:20.595780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] MFA Rules not processed for user `74fa7242b82941cc9ef9f21f122b1e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:20.597948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:20.680644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1096/2189] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:23.080597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:23.082151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:23.082939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:23.191253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] MFA Rules not processed for user `d8c0e96bc0584d8b9be73ae920bbb055`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:23.197833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:23.247950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:23.470654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1094/2190] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:23.471194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:23.487827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b8ccc-0dce-4782-be45-e9f0eae4dccb None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:23.510011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:23.512487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:23.512487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:23.513260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:23.516059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:23.520591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:23.584407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1097/2191] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:07:23 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:07:23.586796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:23.605340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b24c-0a73-425c-8062-7d0867503580 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:23.739663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:23.739819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:23.740321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:23.740901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:23.742093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:23.745142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:23.809053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1095/2192] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:07:23 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:07:23.813739 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:23.842662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcd8ab-ba88-487c-8f36-7add8124a959 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:23.850332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:23.851582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:23.852154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:23.852831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:23.855386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:23.863519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:23.892191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1098/2193] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:07:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:07:23.897862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:23.913193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca9403-0f62-4d94-994a-8db34fe84783 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:23.922954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:23.924195 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:23.924776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:23.925209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] PATH_INFO: `/v3/projects/bce529b9ed10456da8a4e59d2826991f/users/ad280dac47e243db80bc149b9ac1a0c2/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:23.951443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce529b9ed10456da8a4e59d2826991f, user_id=ad280dac47e243db80bc149b9ac1a0c2, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:23.954216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:23.999661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1096/2194] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:07:23 2022] PUT /identity/v3/projects/bce529b9ed10456da8a4e59d2826991f/users/ad280dac47e243db80bc149b9ac1a0c2/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:07:24.004136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:24.020145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb55d93-66c4-4290-ae28-6e992f97ce9c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:24.112968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a170f-58de-42ef-b658-b1abd9866be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:24.113809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a170f-58de-42ef-b658-b1abd9866be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:24.114158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a170f-58de-42ef-b658-b1abd9866be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:24.114699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a170f-58de-42ef-b658-b1abd9866be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:24.115952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a170f-58de-42ef-b658-b1abd9866be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:24.119415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a170f-58de-42ef-b658-b1abd9866be1 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:24.138891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1099/2195] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:07:24.142757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:24.168000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559eac7-6a45-415c-b33a-2f5b31db2543 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:24.182569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42e0e6-b334-4cca-9dbd-80a8d6f505fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:24.183583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42e0e6-b334-4cca-9dbd-80a8d6f505fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:24.183969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42e0e6-b334-4cca-9dbd-80a8d6f505fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:24.184556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42e0e6-b334-4cca-9dbd-80a8d6f505fa admin admin] PATH_INFO: `/v3/projects/bce529b9ed10456da8a4e59d2826991f/users/ad280dac47e243db80bc149b9ac1a0c2/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:24.187604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42e0e6-b334-4cca-9dbd-80a8d6f505fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce529b9ed10456da8a4e59d2826991f, user_id=ad280dac47e243db80bc149b9ac1a0c2, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:24.191076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42e0e6-b334-4cca-9dbd-80a8d6f505fa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:24.210236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1097/2196] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:07:24 2022] PUT /identity/v3/projects/bce529b9ed10456da8a4e59d2826991f/users/ad280dac47e243db80bc149b9ac1a0c2/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:07:24.533674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:24.534221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:24.534512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:24.570423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] MFA Rules not processed for user `d0e5dae087184c56b1cb72fb7705d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:24.572320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:24.595714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:24.725717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1100/2197] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:07:25.515518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee5db1-e6dc-48b7-a697-8a869d30ca34 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:25.516311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee5db1-e6dc-48b7-a697-8a869d30ca34 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:25.516430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee5db1-e6dc-48b7-a697-8a869d30ca34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:25.555670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-56ee5db1-e6dc-48b7-a697-8a869d30ca34 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:25.557702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-56ee5db1-e6dc-48b7-a697-8a869d30ca34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:25.726470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1098/2198] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:07:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:25.736663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:25.738801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:25.886579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f744cb94-9137-43f0-ba47-569822b37a7f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:25.898588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:25.899162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:25.899581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:25.900202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:25.916871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:25.918718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:25.930759 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1101/2199] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:07:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:07:33.599849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b9f1c-7697-4ba2-b855-e54949677ae6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:33.600555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b9f1c-7697-4ba2-b855-e54949677ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:33.600555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b9f1c-7697-4ba2-b855-e54949677ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:33.617327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:33.672332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-936b9f1c-7697-4ba2-b855-e54949677ae6 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:33.681591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-936b9f1c-7697-4ba2-b855-e54949677ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:33.707079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1099/2200] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:07:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:33.766917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:33.782046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4a80-d581-40ba-b7d4-04a7a8908c1d service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:33.817157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:33.818066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:33.819364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:33.819852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:33.845874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:33.847761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:33.867794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1102/2201] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:07:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:07:39.716929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3b8b7-69ab-493c-b17e-b7119c9415e6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:39.717583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3b8b7-69ab-493c-b17e-b7119c9415e6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:39.717583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b3b8b7-69ab-493c-b17e-b7119c9415e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:39.847502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-53b3b8b7-69ab-493c-b17e-b7119c9415e6 None None] MFA Rules not processed for user `ad280dac47e243db80bc149b9ac1a0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:39.849523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-53b3b8b7-69ab-493c-b17e-b7119c9415e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:39.997643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1100/2202] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:40.595285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:07:40.597364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:40.765786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19b5aa-5eb8-4693-8833-45698fa12a29 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:07:40.771908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c39f79-b79e-4b4c-8249-5953dc45c7c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:07:40.772661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c39f79-b79e-4b4c-8249-5953dc45c7c1 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:40.778983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c39f79-b79e-4b4c-8249-5953dc45c7c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:40.779362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c39f79-b79e-4b4c-8249-5953dc45c7c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:40.828764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:40.863872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c39f79-b79e-4b4c-8249-5953dc45c7c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:07:40.865601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c39f79-b79e-4b4c-8249-5953dc45c7c1 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:07:40.889538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1103/2203] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:07:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:07:46.045740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03bfe-4c9c-4852-a119-0bdfaf8f3b29 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:46.046445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03bfe-4c9c-4852-a119-0bdfaf8f3b29 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:46.046445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b03bfe-4c9c-4852-a119-0bdfaf8f3b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:46.098294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-26b03bfe-4c9c-4852-a119-0bdfaf8f3b29 None None] MFA Rules not processed for user `d8c0e96bc0584d8b9be73ae920bbb055`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:46.100623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-26b03bfe-4c9c-4852-a119-0bdfaf8f3b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:46.112419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1101/2204] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:07:50.872737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=13 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:07:51.495063 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:07:51.495353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:07:51.495775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:07:51.551340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] MFA Rules not processed for user `19643be9952649aba81e61d5d55290c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:07:51.553849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:07:51.807702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1104/2205] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:07:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:07:56.077662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=37 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:01.789191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=25 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:02.649389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ecf06-28c4-4e25-8310-3bf5f27ff263 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:02.649906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ecf06-28c4-4e25-8310-3bf5f27ff263 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:02.650105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ecf06-28c4-4e25-8310-3bf5f27ff263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:02.674920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fa5ecf06-28c4-4e25-8310-3bf5f27ff263 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:02.676237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa5ecf06-28c4-4e25-8310-3bf5f27ff263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:02.685088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1102/2206] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:08:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:02.700425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:02.714859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897a82-a9a4-4393-b23f-7f8d487f5483 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:02.715143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897a82-a9a4-4393-b23f-7f8d487f5483 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:02.715415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b897a82-a9a4-4393-b23f-7f8d487f5483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:02.715995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc6342-bed1-430a-9cf9-3ccbbddac360 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:02.721464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:02.722070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:02.722345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:02.722614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:02.732957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:02.734556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:02.737786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-5b897a82-a9a4-4393-b23f-7f8d487f5483 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:02.739069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5b897a82-a9a4-4393-b23f-7f8d487f5483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:02.744443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1105/2207] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:02.750747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1103/2208] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:08:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:08:02.765381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:02.779038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b6e79-9caf-4087-bc95-f41a71d51b65 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:02.792193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:02.793752 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:02.794298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:02.794619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:02.805309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:02.806881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:02.823178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1106/2209] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:03.760374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c5bb-40bb-419a-9ba7-b8173844da74 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:03.760374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c5bb-40bb-419a-9ba7-b8173844da74 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:03.760374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c5bb-40bb-419a-9ba7-b8173844da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:03.785838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c4b8c5bb-40bb-419a-9ba7-b8173844da74 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:03.787442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c4b8c5bb-40bb-419a-9ba7-b8173844da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:03.812083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1104/2210] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 11:08:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:03.821369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:03.837087 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71742c-39cd-4102-94d3-3ddd3d1062c6 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:03.847938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:03.848748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:03.849190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:03.849498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:03.869992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:03.872227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:03.890020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1107/2211] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:04.945628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092aca-9b88-4218-a039-a3f9a6e5f4a2 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:04.946753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092aca-9b88-4218-a039-a3f9a6e5f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:04.947296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42092aca-9b88-4218-a039-a3f9a6e5f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:04.974858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-42092aca-9b88-4218-a039-a3f9a6e5f4a2 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:04.977547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-42092aca-9b88-4218-a039-a3f9a6e5f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:04.985975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1105/2212] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 11:08:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:05.002751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:05.021976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b9cf0-f0f8-4965-909b-3990eba37086 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:05.029998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:05.030624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:05.031037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:05.031346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:05.041896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:05.043504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:05.057960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1108/2213] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:05.373972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a560f-8d2a-4e75-b7dc-5faf363b40b9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:05.374577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a560f-8d2a-4e75-b7dc-5faf363b40b9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:05.375199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a560f-8d2a-4e75-b7dc-5faf363b40b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:05.399975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d52a560f-8d2a-4e75-b7dc-5faf363b40b9 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:05.401607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d52a560f-8d2a-4e75-b7dc-5faf363b40b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:05.410519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1106/2214] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:08:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:08:05.427232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:05.440634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3d65-7fe1-4692-beb1-066a34481ce4 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:05.446403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:05.447020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:05.447295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:05.447570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:05.458585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:05.460444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:05.470127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1109/2215] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:06.127504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:06.128395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:06.128825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:06.155168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:06.156844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:06.172155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1107/2216] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 11:08:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:06.181420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:06.198561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba5081-76b1-4eb2-9907-93d1a2d3d3a9 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:06.218394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:06.219332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:06.219594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:06.219949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:06.242641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:06.246648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:06.259642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1110/2217] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:06.977487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:06.979188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:07.163912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3104d12d-d122-40ca-8cd1-2062ab40a711 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:07.175814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:07.176575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:07.178936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:07.179366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] PATH_INFO: `/v3/users/fe027aad062342688c587428f2bfbfe7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:07.181926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe027aad062342688c587428f2bfbfe7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:07.187801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:07.295166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe027aad062342688c587428f2bfbfe7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:07.295950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe027aad062342688c587428f2bfbfe7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:07.318148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe027aad062342688c587428f2bfbfe7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:07.346600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe027aad062342688c587428f2bfbfe7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:07.361508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-825eed3c-f739-441b-8a48-2244ecae50d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe027aad062342688c587428f2bfbfe7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:07.378322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1108/2218] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:08:06 2022] DELETE /identity/v3/users/fe027aad062342688c587428f2bfbfe7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:07.737517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:07.751749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40327af9-e096-4688-b45d-0b8af3bc0c78 service swift] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:07.930284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:07.930945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:07.931226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:07.931495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] PATH_INFO: `/v3/projects/b2365f1500c5405aab2ea6e892fbb832` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:07.937906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2365f1500c5405aab2ea6e892fbb832)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:07.941058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:08.092709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2365f1500c5405aab2ea6e892fbb832', 'request_id': 'req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:08.113811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2365f1500c5405aab2ea6e892fbb832', 'request_id': 'req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:08.136275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2365f1500c5405aab2ea6e892fbb832', 'request_id': 'req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:08.174949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] The token cache is being invalidate because project b2365f1500c5405aab2ea6e892fbb832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:08:08.175605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:08.178810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-34d0cf4c-923f-4aa9-ad9d-9ede2a532d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:08.183238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1111/2219] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:08:07 2022] DELETE /identity/v3/projects/b2365f1500c5405aab2ea6e892fbb832 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:08.188494 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.189413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.189869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.227600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:08.228913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:08.309133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:08.372102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1109/2220] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:08.379847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.380443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.380929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.422257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:08.423817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:08.443260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1112/2221] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:08.451424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:08.502650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e7e14-f2c9-45fe-9912-6a85159586de None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:08.514435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:08.515133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.515508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.515818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.516527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:08:08.519694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:08.522000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:08.556524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1110/2222] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:08:08.560943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:08.566966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:08.580241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3719e1-9ada-4754-a64e-2bfea21f0aa0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:08.594203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:08.595146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.595614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.596529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.600494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:08.605427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:08.610621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:08.613447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:08.671278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1113/2223] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:08:08 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:08:08.675420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:08.801640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0de9a2-187e-4260-afc6-e568ea212f1d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:08.847247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981c6f0-c54a-46ba-be9b-c19dc324c14a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:08.854470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:08.855119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.855401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.855678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.856423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:08.858942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:08.892143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1114/2224] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:08:08 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:08:08.893124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:08.907568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8fbf1-46b8-459e-bda7-985059d54b48 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:08.914352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:08.914895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.915182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.915460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.916424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:08.922690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:08.930450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:08.931085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.931371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.931648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.940683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1115/2225] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:08:08.941336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:08.941649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:08.954458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbb3987-10b4-491e-9f56-92b27f4ce36a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:08.959683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:08.960344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:08.961034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:08.961326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:08.962004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:08.964468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:08.979586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1116/2226] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:08:08.980589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:09.008191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24e05e-f089-4313-9978-ced0c5467709 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:09.017127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde93a0-6104-4edf-bdbc-5ea8d399a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:09.017532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde93a0-6104-4edf-bdbc-5ea8d399a8d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:09.018155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde93a0-6104-4edf-bdbc-5ea8d399a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:09.018155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde93a0-6104-4edf-bdbc-5ea8d399a8d3 admin admin] PATH_INFO: `/v3/projects/571850583d934ba39350ba72266e9274/users/269d28f3e2504fbdaf3630d4c3373410/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:09.041528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde93a0-6104-4edf-bdbc-5ea8d399a8d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=571850583d934ba39350ba72266e9274, user_id=269d28f3e2504fbdaf3630d4c3373410, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:09.044559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfde93a0-6104-4edf-bdbc-5ea8d399a8d3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:09.073418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1117/2227] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:08:08 2022] PUT /identity/v3/projects/571850583d934ba39350ba72266e9274/users/269d28f3e2504fbdaf3630d4c3373410/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:09.084252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:09.084665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:09.085673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:09.097705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:09.099927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:09.117794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1111/2228] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:08:09.125642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] MFA Rules not processed for user `269d28f3e2504fbdaf3630d4c3373410`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:09.128060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:09.258730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1118/2229] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:09.267106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:09.268739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:09.337503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:09.339212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:09.358140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:09.387834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:09.418116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91d6f74-fcbd-4976-8445-3a760a2c4b7c service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:09.500178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a642d-db88-46ed-9262-8634ca469db5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:09.506790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:09.507967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:09.508484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:09.508785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] PATH_INFO: `/v3/users/ce412ceead11449aa5969320405e9f3a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:09.510732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce412ceead11449aa5969320405e9f3a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:09.512888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:09.542194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5c064-e633-4733-a306-5ffc73909902 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:09.543119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c064-e633-4733-a306-5ffc73909902 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:09.543890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c064-e633-4733-a306-5ffc73909902 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:09.544608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5c064-e633-4733-a306-5ffc73909902 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:09.559800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5c064-e633-4733-a306-5ffc73909902 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:09.562167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5c064-e633-4733-a306-5ffc73909902 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:09.597174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1112/2230] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:09.598001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce412ceead11449aa5969320405e9f3a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:09.613568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce412ceead11449aa5969320405e9f3a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:09.626312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce412ceead11449aa5969320405e9f3a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:09.662570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce412ceead11449aa5969320405e9f3a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:09.666496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce412ceead11449aa5969320405e9f3a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:09.685036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1119/2231] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:08:09 2022] DELETE /identity/v3/users/ce412ceead11449aa5969320405e9f3a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:08:09.688878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:09.689639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:09.690181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:09.745372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:09.748587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:09.875958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1113/2232] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:09.898837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:09.900541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:10.026697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b6c4c-97da-431d-aa1e-028c23e0870c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:10.031079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbe936-97cf-463b-a8c7-d3cd38fe9732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:10.031630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbe936-97cf-463b-a8c7-d3cd38fe9732 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:10.031911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbe936-97cf-463b-a8c7-d3cd38fe9732 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:10.032210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbe936-97cf-463b-a8c7-d3cd38fe9732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:10.060070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbe936-97cf-463b-a8c7-d3cd38fe9732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:10.061548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbe936-97cf-463b-a8c7-d3cd38fe9732 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:10.081146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1120/2233] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:10.771277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:10.772891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:10.792119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4950f-ebe8-4102-8f7e-6a568599fc4a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:10.819021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:10.823308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:10.824607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:10.824903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] PATH_INFO: `/v3/projects/48c48ea6637c49d7b7cec6e805c4be87` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:10.827552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48c48ea6637c49d7b7cec6e805c4be87)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:10.831409 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:10.955408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48c48ea6637c49d7b7cec6e805c4be87', 'request_id': 'req-000bdd15-00b6-4475-8cfd-10d629c4d5b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:10.966808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48c48ea6637c49d7b7cec6e805c4be87', 'request_id': 'req-000bdd15-00b6-4475-8cfd-10d629c4d5b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:10.987289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48c48ea6637c49d7b7cec6e805c4be87', 'request_id': 'req-000bdd15-00b6-4475-8cfd-10d629c4d5b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:11.000593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] The token cache is being invalidate because project 48c48ea6637c49d7b7cec6e805c4be87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:08:11.000884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:11.001705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-000bdd15-00b6-4475-8cfd-10d629c4d5b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:11.003677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1114/2234] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:08:10 2022] DELETE /identity/v3/projects/48c48ea6637c49d7b7cec6e805c4be87 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:11.013103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d755-265c-443a-b46c-b741af0660eb None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.013103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d755-265c-443a-b46c-b741af0660eb None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.013103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576d755-265c-443a-b46c-b741af0660eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.040052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1576d755-265c-443a-b46c-b741af0660eb None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:11.041326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1576d755-265c-443a-b46c-b741af0660eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:11.101236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:11.178893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1121/2235] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:11.186473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.186984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.187640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.239953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:11.247916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:11.292077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1115/2236] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:11.293351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d755-265c-443a-b46c-b741af0660eb None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:11.311217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576d755-265c-443a-b46c-b741af0660eb None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:11.323150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:11.324140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.324621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.325083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.325938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:08:11.331433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:11.333436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:11.361915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1122/2237] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:08:11.368046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:11.385778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324bbf37-aedf-4ab0-807b-50f0ac969b50 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:11.410343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:11.411665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.411665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.411927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.413315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:11.418279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:11.431156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:11.505077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1116/2238] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:08:11 2022] POST /identity/v3/projects => generated 397 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:08:11.510515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:11.526331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428369f9-ae7f-4809-8d9f-be15e08b1842 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:11.657122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:11.658497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.658945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.659585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.660498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:11.663215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:11.713902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1123/2239] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:08:11 2022] POST /identity/v3/users => generated 299 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:08:11.717617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:11.732213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48471b2d-6afb-494f-a52e-a734710966d4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:11.754906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:11.759016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.760261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.760261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.764482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:11.802027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:11.818027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1117/2240] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:08:11.822679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:11.837239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952fe653-f8fc-45cd-b954-35aa98bfac0b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:11.875713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd0ce3b-32a4-4345-8709-b6fa7bd9a353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:11.876377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0ce3b-32a4-4345-8709-b6fa7bd9a353 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.877018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0ce3b-32a4-4345-8709-b6fa7bd9a353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.880895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd0ce3b-32a4-4345-8709-b6fa7bd9a353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.882997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0ce3b-32a4-4345-8709-b6fa7bd9a353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:11.890497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd0ce3b-32a4-4345-8709-b6fa7bd9a353 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:11.917287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1124/2241] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:08:11.921551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:11.939478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba97d0-0ffb-43b2-a806-4cd74486e6a6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:11.954358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:11.955052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:11.955332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:11.955607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] PATH_INFO: `/v3/projects/2d10792dc0214dc99520659405d156ef/users/3e185f170cfb4457adff9ba8ca8d2f3b/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:11.999438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d10792dc0214dc99520659405d156ef, user_id=3e185f170cfb4457adff9ba8ca8d2f3b, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:12.002828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:12.046006 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:12.047743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1118/2242] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:08:11 2022] PUT /identity/v3/projects/2d10792dc0214dc99520659405d156ef/users/3e185f170cfb4457adff9ba8ca8d2f3b/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:12.048986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:12.049270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:12.118010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] MFA Rules not processed for user `3e185f170cfb4457adff9ba8ca8d2f3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:12.119296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:12.335203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1125/2243] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:12 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:12.344858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:12.347116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:12.592072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a0720-34b5-44dd-bebc-e96b2246f49b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:12.599022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a9447-5841-4931-8acf-a495b7262032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:12.600175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9447-5841-4931-8acf-a495b7262032 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:12.600684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9447-5841-4931-8acf-a495b7262032 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:12.602425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a9447-5841-4931-8acf-a495b7262032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:12.614026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a9447-5841-4931-8acf-a495b7262032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:12.616334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a9447-5841-4931-8acf-a495b7262032 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:12.638124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1119/2244] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:22.048410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:22.095780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:22.136186 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04aa52-539f-4b4b-9275-ca3cc7807632 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:22.141232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:22.141884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:22.142171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:22.142449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] PATH_INFO: `/v3/users/3e185f170cfb4457adff9ba8ca8d2f3b` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:22.143618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e185f170cfb4457adff9ba8ca8d2f3b)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:22.145609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:22.197556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e185f170cfb4457adff9ba8ca8d2f3b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:22.204667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e185f170cfb4457adff9ba8ca8d2f3b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:22.213840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e185f170cfb4457adff9ba8ca8d2f3b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:22.233387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e185f170cfb4457adff9ba8ca8d2f3b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:22.234138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e185f170cfb4457adff9ba8ca8d2f3b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:22.241051 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1126/2245] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:08:22 2022] DELETE /identity/v3/users/3e185f170cfb4457adff9ba8ca8d2f3b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:22.245613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:22.246133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:22.246274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:22.270961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:22.272762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:22.338338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1120/2246] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:08:22.348657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:22.350357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:22.424408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdabaaf3-b3b6-40da-b29a-88d5cf7a4adb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:22.429042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:22.429753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:22.430042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:22.430322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:22.440195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:22.441658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:22.452832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1127/2247] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:23.157076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:23.159072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:23.173165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40de157b-6587-4026-93bc-3b90477c3d49 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:23.180407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:23.181249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:23.181681 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:23.182095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] PATH_INFO: `/v3/projects/2d10792dc0214dc99520659405d156ef` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:23.183415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d10792dc0214dc99520659405d156ef)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:23.185595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:23.254843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d10792dc0214dc99520659405d156ef', 'request_id': 'req-f4eebeb3-5ae4-4574-9323-16ab89f35e42'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:23.257363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:23.268546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d10792dc0214dc99520659405d156ef', 'request_id': 'req-f4eebeb3-5ae4-4574-9323-16ab89f35e42'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:23.275936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d10792dc0214dc99520659405d156ef', 'request_id': 'req-f4eebeb3-5ae4-4574-9323-16ab89f35e42'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:23.287790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] The token cache is being invalidate because project 2d10792dc0214dc99520659405d156ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:08:23.288147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:23.288779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f4eebeb3-5ae4-4574-9323-16ab89f35e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:23.290414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1121/2248] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:08:23 2022] DELETE /identity/v3/projects/2d10792dc0214dc99520659405d156ef => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:24.043569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:24.045340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:24.161865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:24.192676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f889ea-797b-49b0-9db8-70809530d42c service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:24.314110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:24.314906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:24.315288 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:24.315818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:24.319260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:24.340685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:24.343907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b75ac-f5ff-4bc4-aab0-8b8070b86964 service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:24.362628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1128/2249] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:26.429406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0dbf0-a858-4032-99c3-95c56966c312 None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.430021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0dbf0-a858-4032-99c3-95c56966c312 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.430021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0dbf0-a858-4032-99c3-95c56966c312 None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.432203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1122/2250] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:08:26.440549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012fd4-e7fd-49de-87c3-0532f0e88642 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.441708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012fd4-e7fd-49de-87c3-0532f0e88642 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.442147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43012fd4-e7fd-49de-87c3-0532f0e88642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.466580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-43012fd4-e7fd-49de-87c3-0532f0e88642 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:26.467915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-43012fd4-e7fd-49de-87c3-0532f0e88642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:26.542646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1129/2251] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:26.556590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0dbf0-a858-4032-99c3-95c56966c312 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:26.558432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-44f0dbf0-a858-4032-99c3-95c56966c312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:26.623548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f0dbf0-a858-4032-99c3-95c56966c312 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:26.627568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af59176b-4d14-4f82-9f36-79e0beea436d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:26.628203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59176b-4d14-4f82-9f36-79e0beea436d service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.628481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59176b-4d14-4f82-9f36-79e0beea436d service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.628750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af59176b-4d14-4f82-9f36-79e0beea436d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.639393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59176b-4d14-4f82-9f36-79e0beea436d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:26.640865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af59176b-4d14-4f82-9f36-79e0beea436d service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:26.652948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1123/2252] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:26.741165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9e03f-722c-49a2-8ced-8d6128c73f97 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.741458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9e03f-722c-49a2-8ced-8d6128c73f97 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.742236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9e03f-722c-49a2-8ced-8d6128c73f97 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.743856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1130/2253] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:08:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:08:26.749064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901eccb-b7d7-4dae-9077-b875ee89707f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.749462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901eccb-b7d7-4dae-9077-b875ee89707f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.749805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901eccb-b7d7-4dae-9077-b875ee89707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.770427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3901eccb-b7d7-4dae-9077-b875ee89707f None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:26.771831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3901eccb-b7d7-4dae-9077-b875ee89707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:26.826951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1124/2254] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:26.839706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9e03f-722c-49a2-8ced-8d6128c73f97 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:26.841446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-96c9e03f-722c-49a2-8ced-8d6128c73f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:26.856065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9e03f-722c-49a2-8ced-8d6128c73f97 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:26.859900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1627b4c-4066-4826-847e-36e1ec6910ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:26.860524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627b4c-4066-4826-847e-36e1ec6910ce service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.860802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627b4c-4066-4826-847e-36e1ec6910ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.861074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1627b4c-4066-4826-847e-36e1ec6910ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.869885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627b4c-4066-4826-847e-36e1ec6910ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:26.871514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1627b4c-4066-4826-847e-36e1ec6910ce service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:26.880467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1131/2255] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:26.966253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f141d32-a255-455b-87e8-d077ca1defee None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.966549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f141d32-a255-455b-87e8-d077ca1defee None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.966831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f141d32-a255-455b-87e8-d077ca1defee None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.968276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1125/2256] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:08:26.973396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede19d3-2ba2-4a51-9808-de9d17bddf59 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:26.973680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede19d3-2ba2-4a51-9808-de9d17bddf59 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:26.973955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede19d3-2ba2-4a51-9808-de9d17bddf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:26.994740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-aede19d3-2ba2-4a51-9808-de9d17bddf59 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:26.995929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-aede19d3-2ba2-4a51-9808-de9d17bddf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:27.005321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1132/2257] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:08:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:27.015753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f141d32-a255-455b-87e8-d077ca1defee None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:27.027444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f141d32-a255-455b-87e8-d077ca1defee None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:27.031115 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:27.031704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:27.031983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:27.032280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:27.043578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:27.045062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:27.055880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1126/2258] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:31.126241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:31.126733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:31.126872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:31.146744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:31.147857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:31.207556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1133/2259] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:08:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:31.219396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:31.220879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:31.294950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e503e16c-71d7-4039-a075-bcd7e0b82e34 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:31.300407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f3b6-a110-493d-a78a-de36682ac9aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:31.301042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f3b6-a110-493d-a78a-de36682ac9aa service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:31.301325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f3b6-a110-493d-a78a-de36682ac9aa service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:31.301601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f3b6-a110-493d-a78a-de36682ac9aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:31.313044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7f3b6-a110-493d-a78a-de36682ac9aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:31.314641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7f3b6-a110-493d-a78a-de36682ac9aa service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:31.323743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1127/2260] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:08:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:08:41.200281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:41.320166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:42.499603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:42.501093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:42.578695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2eff4-a9e1-4467-84dd-9f540e2c839e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:42.583256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:42.583882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:42.584204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:42.584549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] PATH_INFO: `/v3/users/269d28f3e2504fbdaf3630d4c3373410` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:42.585618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269d28f3e2504fbdaf3630d4c3373410)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:42.587520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:42.634621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269d28f3e2504fbdaf3630d4c3373410'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:42.641762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269d28f3e2504fbdaf3630d4c3373410'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:42.649816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269d28f3e2504fbdaf3630d4c3373410'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:42.662153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269d28f3e2504fbdaf3630d4c3373410'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:42.662799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269d28f3e2504fbdaf3630d4c3373410'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:42.669104 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1134/2261] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:08:42 2022] DELETE /identity/v3/users/269d28f3e2504fbdaf3630d4c3373410 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:42.673754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:42.674261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:42.674398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:42.694387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:42.695605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:42.753812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1128/2262] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:42.764320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:42.765833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:42.823775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379790a-a680-4cd9-af93-86c3eb8e00f5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:42.827503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30038b-4b17-434e-b3f7-6aa413e59998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:42.828133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30038b-4b17-434e-b3f7-6aa413e59998 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:42.828417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30038b-4b17-434e-b3f7-6aa413e59998 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:42.828692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30038b-4b17-434e-b3f7-6aa413e59998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:42.838153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30038b-4b17-434e-b3f7-6aa413e59998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:42.839584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30038b-4b17-434e-b3f7-6aa413e59998 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:42.849402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1135/2263] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:43.123736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.125465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:43.138894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5ddf3-b6c5-4cf4-8a85-b38c1e4f8a29 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.143616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.144277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.144559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.144832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] PATH_INFO: `/v3/projects/571850583d934ba39350ba72266e9274` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.146087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=571850583d934ba39350ba72266e9274)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.148442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.195851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '571850583d934ba39350ba72266e9274', 'request_id': 'req-bebdf29b-a798-4296-9686-2dc8526adf0c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:43.203511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '571850583d934ba39350ba72266e9274', 'request_id': 'req-bebdf29b-a798-4296-9686-2dc8526adf0c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:43.210030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '571850583d934ba39350ba72266e9274', 'request_id': 'req-bebdf29b-a798-4296-9686-2dc8526adf0c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:43.221759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] The token cache is being invalidate because project 571850583d934ba39350ba72266e9274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:08:43.222049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:43.222718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bebdf29b-a798-4296-9686-2dc8526adf0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:43.225107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1129/2264] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:08:43 2022] DELETE /identity/v3/projects/571850583d934ba39350ba72266e9274 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:43.230455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.230767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.231127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.253737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:43.255059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:43.272746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:43.319518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1136/2265] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:43.328439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.328715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.328988 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.353770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:43.354999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:43.363125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1130/2266] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:43.367537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.377396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b43005-4c10-4951-a08a-8a1c8a6051d0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.382238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.382871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.383150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.383422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.384099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:08:43.384407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.385995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.402641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1137/2267] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:08:43.407383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.417508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e8a83-563b-4f47-8e70-fb740166178f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.422422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.423167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.423321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.423578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.424247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.426289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.458715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1131/2268] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:08:43 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:08:43.463180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.474795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a2f13-69a2-46ee-90af-c74a49e123dc admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.547422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.548139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.548391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.548657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.549296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.551190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.573564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1138/2269] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:08:43 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:08:43.577677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.588779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edbacd8-ff21-4582-a94c-03d385c3f2d8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.593838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.594525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.594865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.595205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.595895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.597713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.612051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1132/2270] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:08:43.613947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.627454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6292039-f2cb-4a7d-80ce-f2cdc7ce8f8a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.633140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a36bb-a93d-4ef0-8a7d-ab8c9ab5af4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.633745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a36bb-a93d-4ef0-8a7d-ab8c9ab5af4b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.634027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a36bb-a93d-4ef0-8a7d-ab8c9ab5af4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.634302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a36bb-a93d-4ef0-8a7d-ab8c9ab5af4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.634953 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a36bb-a93d-4ef0-8a7d-ab8c9ab5af4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.636590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a36bb-a93d-4ef0-8a7d-ab8c9ab5af4b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.649340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1139/2271] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:08:43.653713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.667442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99e293f-bd75-4234-9260-d0e19997f782 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.672402 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.673149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.673496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.673935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] PATH_INFO: `/v3/projects/6c2f1d1a3f4c4ae9bbb6534c1b017854/users/9fa78be599d24ce1bc2c31f9ce8aa70f/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.692599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2f1d1a3f4c4ae9bbb6534c1b017854, user_id=9fa78be599d24ce1bc2c31f9ce8aa70f, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.696289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.711311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1133/2272] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:08:43 2022] PUT /identity/v3/projects/6c2f1d1a3f4c4ae9bbb6534c1b017854/users/9fa78be599d24ce1bc2c31f9ce8aa70f/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:43.721772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.722062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.722337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.750636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] MFA Rules not processed for user `9fa78be599d24ce1bc2c31f9ce8aa70f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:43.751833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:43.819650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1140/2273] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:43.831401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.833912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:43.855027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:43.891251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f850bc-aefd-4b37-b20c-4eff9cb1f57f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:43.894869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c183d64-1bcf-4583-bbac-6600d5ed03ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:43.896031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183d64-1bcf-4583-bbac-6600d5ed03ba service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:43.896309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183d64-1bcf-4583-bbac-6600d5ed03ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:43.896575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c183d64-1bcf-4583-bbac-6600d5ed03ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:43.904846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c183d64-1bcf-4583-bbac-6600d5ed03ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:43.906288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c183d64-1bcf-4583-bbac-6600d5ed03ba service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:43.914887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1134/2274] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:43.939671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:43.959593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:44.002692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb4bb9-889c-4b87-ad54-2ab10c4112d0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:44.008337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:44.008980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:44.009264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:44.009543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] PATH_INFO: `/v3/users/9fa78be599d24ce1bc2c31f9ce8aa70f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:44.010692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa78be599d24ce1bc2c31f9ce8aa70f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:44.012652 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:44.061163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa78be599d24ce1bc2c31f9ce8aa70f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.068168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa78be599d24ce1bc2c31f9ce8aa70f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.076637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa78be599d24ce1bc2c31f9ce8aa70f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.090858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa78be599d24ce1bc2c31f9ce8aa70f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.091796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa78be599d24ce1bc2c31f9ce8aa70f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.098062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1141/2275] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:08:43 2022] DELETE /identity/v3/users/9fa78be599d24ce1bc2c31f9ce8aa70f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:44.103389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:44.103677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:44.103950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:44.122965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:08:44.124203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:44.187731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1135/2276] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:08:44.197461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:44.198864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:44.258443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd7231-11d5-4240-9d66-0a6d98fe67b8 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:44.262318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:44.262938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:44.263203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:44.263499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:44.273469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:44.274906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:44.284556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1142/2277] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:44.591105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:44.592574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:44.610884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd0302-9cb0-4475-8556-ac6edb9a9c0e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:44.616420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:44.617185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:44.617559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:44.617963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] PATH_INFO: `/v3/projects/6c2f1d1a3f4c4ae9bbb6534c1b017854` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:44.619260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2f1d1a3f4c4ae9bbb6534c1b017854)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:44.621297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:44.675705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2f1d1a3f4c4ae9bbb6534c1b017854', 'request_id': 'req-2da392f5-eb1d-4376-be49-9279ec279fe9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.684527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2f1d1a3f4c4ae9bbb6534c1b017854', 'request_id': 'req-2da392f5-eb1d-4376-be49-9279ec279fe9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.690372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2f1d1a3f4c4ae9bbb6534c1b017854', 'request_id': 'req-2da392f5-eb1d-4376-be49-9279ec279fe9'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.701768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] The token cache is being invalidate because project 6c2f1d1a3f4c4ae9bbb6534c1b017854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:08:44.702060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.702713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2da392f5-eb1d-4376-be49-9279ec279fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:08:44.704549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1136/2278] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:08:44 2022] DELETE /identity/v3/projects/6c2f1d1a3f4c4ae9bbb6534c1b017854 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:08:45.106523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:08:45.115926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53617b37-ea5d-4497-a58f-e64a6e9a0b23 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:08:45.155341 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:45.168624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:08:45.169227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:08:45.169509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:08:45.169784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:08:45.171513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:08:45.234296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:08:45.235738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:08:45.245457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1143/2279] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:08:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:08:54.698680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:08:55.241041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=32 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:01.544313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd3e0-d2c7-4025-9f39-cdbd34ca9b89 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:01.544851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd3e0-d2c7-4025-9f39-cdbd34ca9b89 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:01.544851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bd3e0-d2c7-4025-9f39-cdbd34ca9b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:01.582822 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-042bd3e0-d2c7-4025-9f39-cdbd34ca9b89 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:09:01.584404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-042bd3e0-d2c7-4025-9f39-cdbd34ca9b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:01.642898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1137/2280] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 11:09:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:09:01.668603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:09:01.670182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:01.750009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c60cf-ad75-4419-a416-287656eaf988 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:09:01.755420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca2b4a3-0c01-4ea1-a05c-ad594beac3f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:09:01.756066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2b4a3-0c01-4ea1-a05c-ad594beac3f3 service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:01.756351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2b4a3-0c01-4ea1-a05c-ad594beac3f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:01.756658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca2b4a3-0c01-4ea1-a05c-ad594beac3f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:01.765695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2b4a3-0c01-4ea1-a05c-ad594beac3f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:09:01.767308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca2b4a3-0c01-4ea1-a05c-ad594beac3f3 service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:09:01.776122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1144/2281] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:09:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 11:09:11.635948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=25 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:11.772435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=32 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:17.978754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86ac43-ed4b-4d6b-9657-0da66128de27 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:17.979333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86ac43-ed4b-4d6b-9657-0da66128de27 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:17.979333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86ac43-ed4b-4d6b-9657-0da66128de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:17.998462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab86ac43-ed4b-4d6b-9657-0da66128de27 None None] MFA Rules not processed for user `d0e5dae087184c56b1cb72fb7705d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:09:17.999668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab86ac43-ed4b-4d6b-9657-0da66128de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:18.056306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1138/2282] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:09:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:09:28.048764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=25 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:42.479767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:42.480384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:42.480384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:42.500318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] MFA Rules not processed for user `19643be9952649aba81e61d5d55290c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:09:42.502099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:42.562489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1145/2283] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:09:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:09:52.555799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=25 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:09:54.863359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aae33-d148-4503-932c-8268e398959e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:09:54.863905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aae33-d148-4503-932c-8268e398959e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:09:54.863905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558aae33-d148-4503-932c-8268e398959e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:09:54.883853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-558aae33-d148-4503-932c-8268e398959e None None] MFA Rules not processed for user `714872990c384663bcd564cd5b9ae432`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:09:54.885067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-558aae33-d148-4503-932c-8268e398959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:09:54.938939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1139/2284] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:09:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:10:04.931932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=25 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:16.520815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:16.522244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:16.584601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cf1dcd-59ba-4e2a-beb6-f99b413ec7e8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:16.588230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:16.588835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:16.589115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:16.589386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:16.591108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:16.662663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:16.664141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:16.675261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1146/2285] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:20.314297 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aae33-d148-4503-932c-8268e398959e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:20.315728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-558aae33-d148-4503-932c-8268e398959e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:20.327526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558aae33-d148-4503-932c-8268e398959e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:20.331725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:20.332370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:20.332654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:20.332925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:20.334638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:20.348813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:20.350256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:20.361278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1140/2286] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:24.365529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:24.366917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:24.377962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf1579-a9c0-454b-bba7-90173793cac4 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:24.381767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:24.382394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:24.382675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:24.382948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:24.384682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:24.395578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:24.397169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0b84fa-d0d1-4448-9c48-aa2555129b16 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:24.406471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1147/2287] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:28.889726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:28.900397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3088364-2ae7-468e-971c-8e93e8f6f443 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:28.904106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:28.904702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:28.904977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:28.905242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:28.907004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:28.918460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:28.919877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:28.928726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1141/2288] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:28 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:34.402213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=60 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:38.823270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a2de-1d77-436f-b636-64621bf6b5bb None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:38.823949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a2de-1d77-436f-b636-64621bf6b5bb None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:38.823949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a2de-1d77-436f-b636-64621bf6b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:38.848445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e3e0a2de-1d77-436f-b636-64621bf6b5bb None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:10:38.849680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e3e0a2de-1d77-436f-b636-64621bf6b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:38.859308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1148/2289] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 11:10:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:10:38.921597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:38.924456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=12 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:38.935758 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ec88-6cc8-40c1-ad63-ddad3c07a238 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:38.941427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:38.942168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:38.942554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:38.942923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:38.952137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:38.953774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:38.971070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1142/2290] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:10:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:10:42.052300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:42.052998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:42.053365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:42.076129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] MFA Rules not processed for user `d0e5dae087184c56b1cb72fb7705d4d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:10:42.077553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:42.085027 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1149/2291] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:10:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:10:42.686768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:42.688827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:42.701888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f40da-96ca-436b-8115-d1134aba02cc service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:42.706585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:42.707206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:42.707489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:42.707759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:42.709577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:42.719888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:42.721961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:42.731298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1143/2292] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:43.321561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:43.323695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:43.451477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabf53d1-c711-445d-8425-085bc8cfaee3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:43.457115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:43.457679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:43.457963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:43.458242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:43.460134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:43.473222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:43.474856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:43.497259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1150/2293] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:43.707405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:43.708914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:43.727924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa729335-3a8d-4ff5-be11-4392ba8243ef service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:43.736479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:43.738292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:43.738817 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:43.739303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:43.741777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:43.776037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:43.780532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:43.806213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1144/2294] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:10:46.961148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:46.989932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e4ee2-b187-42b8-ac3d-df4347376f35 service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:47.007342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:47.022007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:47.022507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:47.023056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:47.030217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:47.048550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:47.051309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:47.064289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1151/2295] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:48.250117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:48.251693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:48.322861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef534865-e2f2-4acb-be31-ecc0493a741d service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:48.327357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:48.327986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:48.328291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:48.328562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] PATH_INFO: `/v3/users/74fa7242b82941cc9ef9f21f122b1e4c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:48.329680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fa7242b82941cc9ef9f21f122b1e4c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:48.332355 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:48.384537 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fa7242b82941cc9ef9f21f122b1e4c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.385193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fa7242b82941cc9ef9f21f122b1e4c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.394545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fa7242b82941cc9ef9f21f122b1e4c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.408184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fa7242b82941cc9ef9f21f122b1e4c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.415728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fa7242b82941cc9ef9f21f122b1e4c'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.421668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1145/2296] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:10:48 2022] DELETE /identity/v3/users/74fa7242b82941cc9ef9f21f122b1e4c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:48.594349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:48.609982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d9d58-751b-43a6-b5c0-b375697204a6 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:48.669585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:48.670215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:48.670495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:48.670767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] PATH_INFO: `/v3/projects/6e4af9139d2b47cf8ea8646c5a4f4a9f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:48.671881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4af9139d2b47cf8ea8646c5a4f4a9f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:48.673863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:48.719662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4af9139d2b47cf8ea8646c5a4f4a9f', 'request_id': 'req-998a98b4-37b8-4a38-b872-3fa243c82f28'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.726547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4af9139d2b47cf8ea8646c5a4f4a9f', 'request_id': 'req-998a98b4-37b8-4a38-b872-3fa243c82f28'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.732149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4af9139d2b47cf8ea8646c5a4f4a9f', 'request_id': 'req-998a98b4-37b8-4a38-b872-3fa243c82f28'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.734600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:48.745201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] The token cache is being invalidate because project 6e4af9139d2b47cf8ea8646c5a4f4a9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:10:48.745491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.746111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.747601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1152/2297] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:10:48 2022] DELETE /identity/v3/projects/6e4af9139d2b47cf8ea8646c5a4f4a9f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:48.751642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:48.753378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:48.766610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11511b6b-8948-45e2-b97a-c298164b0242 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:48.806877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:48.831577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:48.832377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:48.832658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:48.832938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] PATH_INFO: `/v3/users/6f06f4bf10754a0ca89a0ce4bc266484` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:48.834090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f06f4bf10754a0ca89a0ce4bc266484)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:48.836022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:48.882713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f06f4bf10754a0ca89a0ce4bc266484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.883319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f06f4bf10754a0ca89a0ce4bc266484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.893353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f06f4bf10754a0ca89a0ce4bc266484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.910295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f06f4bf10754a0ca89a0ce4bc266484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.917486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6635541a-9cd6-4de0-b3c8-40fb76302c1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f06f4bf10754a0ca89a0ce4bc266484'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:48.923622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1146/2298] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:10:48 2022] DELETE /identity/v3/users/6f06f4bf10754a0ca89a0ce4bc266484 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:49.097403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.099008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:49.114024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a98b4-37b8-4a38-b872-3fa243c82f28 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.178400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.179156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.179496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.179833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] PATH_INFO: `/v3/projects/3dc2dbf05b5c4b2bb2fb7ad3ddc18f40` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.181704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc2dbf05b5c4b2bb2fb7ad3ddc18f40)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.184102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.242355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc2dbf05b5c4b2bb2fb7ad3ddc18f40', 'request_id': 'req-d94f9a8c-1bc2-4076-a750-d083a07faa67'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:49.248462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc2dbf05b5c4b2bb2fb7ad3ddc18f40', 'request_id': 'req-d94f9a8c-1bc2-4076-a750-d083a07faa67'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:49.254651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc2dbf05b5c4b2bb2fb7ad3ddc18f40', 'request_id': 'req-d94f9a8c-1bc2-4076-a750-d083a07faa67'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:49.267290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] The token cache is being invalidate because project 3dc2dbf05b5c4b2bb2fb7ad3ddc18f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:10:49.267585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:49.268275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d94f9a8c-1bc2-4076-a750-d083a07faa67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:49.270404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1153/2299] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:10:49 2022] DELETE /identity/v3/projects/3dc2dbf05b5c4b2bb2fb7ad3ddc18f40 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:49.275343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.275816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.275899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.296939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:10:49.298142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:49.358715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1147/2300] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:10:49.368371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.368666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.368944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.390473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:10:49.391676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:49.399801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1154/2301] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:10:49.404212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.415384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9328f9dd-b158-457d-9e81-08f4112582e7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.420067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.420663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.420941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.421233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.421858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:10:49.422161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.424457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.436000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1148/2302] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:10:49.441151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.451082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c2b89-c110-4cea-921b-8998dd43b430 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.456384 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.456937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.457219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.457501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.458185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.460272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.490478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1155/2303] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:10:49 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:10:49.494419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.505980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752e446-7b99-48f1-b8c6-74ee173eb0cc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.559861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:49.576221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.576839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.577121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.577400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.578057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.580064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.608352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1149/2304] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:10:49 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:10:49.609659 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.620622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87920d2a-ca63-4dad-880b-4c7c9c92ab1d admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.625257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.625825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.626170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.626372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.627095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.628795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.638991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1156/2305] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:10:49.643157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.653091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bd5447-6067-4e95-bf59-735947430ddb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.657734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49eb97-8ca5-4e48-b48b-8d56cc7fc7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.658834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49eb97-8ca5-4e48-b48b-8d56cc7fc7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.659114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49eb97-8ca5-4e48-b48b-8d56cc7fc7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.659386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49eb97-8ca5-4e48-b48b-8d56cc7fc7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.660051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49eb97-8ca5-4e48-b48b-8d56cc7fc7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.661939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49eb97-8ca5-4e48-b48b-8d56cc7fc7a2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.672520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1150/2306] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:10:49.677608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.687398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988c05c-23a7-4e67-a6a3-5ca52dbc515f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.692516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.693156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.693431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.693701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] PATH_INFO: `/v3/projects/da5b36cb688f49e1bf1cf75a108858fa/users/16696c34de254f3babf538c879f2ce49/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.709911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5b36cb688f49e1bf1cf75a108858fa, user_id=16696c34de254f3babf538c879f2ce49, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.712597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.725223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1157/2307] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:10:49 2022] PUT /identity/v3/projects/da5b36cb688f49e1bf1cf75a108858fa/users/16696c34de254f3babf538c879f2ce49/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:49.730370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.730766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.731121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.749868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:10:49.751061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:49.810122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1151/2308] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:10:49.820415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:49.822242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:49.844423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:49.886977 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3288ef-0798-44e8-8b4b-3c7ccd48ee90 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:49.890741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:49.891283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:49.891561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:49.891831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:49.901202 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:49.902618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:49.912872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1158/2309] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:52.201016 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:52.202556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:52.292597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cab9a-74a0-4798-a1f8-78b451cb3925 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:52.300277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:52.301031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:52.301432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:52.301802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:52.304062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:52.394678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:52.396420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f2e3d-3713-4db4-aafd-77ceb17f3a05 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:52.431093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1152/2310] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:54.417206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:54.428390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2a889-0d54-418f-819c-da5edf22dfb1 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:54.433575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:54.434480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:54.434751 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:54.435183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:54.437228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:54.452134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:54.455910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:54.467324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1159/2311] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:10:55.118417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.120088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.120379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.130878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:55.214659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] MFA Rules not processed for user `16696c34de254f3babf538c879f2ce49`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:10:55.217513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:55.325652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1153/2312] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:10:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:10:55.326946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:55.343358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83f252-535f-46bc-893b-5d6d3dee9131 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:55.350398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:55.351679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.351975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.352269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.352912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:55.364319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:55.407785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1160/2313] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:10:55 2022] POST /identity/v3/projects => generated 377 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:10:55.412754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:55.426449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9fe7c2-4205-4a14-9ae3-1241c748c233 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:55.500650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:55.501695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.502617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.502991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.503816 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:55.506391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:55.537022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1154/2314] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:10:55 2022] POST /identity/v3/users => generated 295 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:10:55.540675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:55.552871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb0a4-1d75-44c2-9804-a846dd64bf4a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:55.558095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:55.558834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.559300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.559710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.560529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:55.562134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:55.574988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1161/2315] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:10:55.580585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:55.594836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82953737-314c-41c1-8b7f-4cc6162179a6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:55.602669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:55.603275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.603552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.603826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] PATH_INFO: `/v3/projects/694830b09d0d4ed0be53202125eb5398/users/8a89943b6692461789bec2e224e845c6/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.624106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=694830b09d0d4ed0be53202125eb5398, user_id=8a89943b6692461789bec2e224e845c6, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:55.626832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:55.642862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1155/2316] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:10:55 2022] PUT /identity/v3/projects/694830b09d0d4ed0be53202125eb5398/users/8a89943b6692461789bec2e224e845c6/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:55.646458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:55.656779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277947db-765e-460d-996d-fc66007b17f5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:55.729267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:55.729903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.730181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.730452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.731180 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:55.733172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:55.753545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1162/2317] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:10:55.757575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:55.773675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e40f8a-cbb5-40c5-9213-2d62317f3234 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:55.790002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:55.790946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:55.792132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:55.793110 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] PATH_INFO: `/v3/projects/694830b09d0d4ed0be53202125eb5398/users/8a89943b6692461789bec2e224e845c6/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:55.795533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=694830b09d0d4ed0be53202125eb5398, user_id=8a89943b6692461789bec2e224e845c6, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:55.800794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:55.814673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1156/2318] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:10:55 2022] PUT /identity/v3/projects/694830b09d0d4ed0be53202125eb5398/users/8a89943b6692461789bec2e224e845c6/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:10:59.641389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:10:59.643601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:10:59.672675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7195ed61-10ff-4918-b4dd-5c89725bacd8 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:10:59.837912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:10:59.838571 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:10:59.838886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:10:59.839163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] PATH_INFO: `/v3/users/19643be9952649aba81e61d5d55290c4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:10:59.842426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19643be9952649aba81e61d5d55290c4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:10:59.844527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:10:59.877387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:10:59.958980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19643be9952649aba81e61d5d55290c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:59.979401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19643be9952649aba81e61d5d55290c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:10:59.996444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19643be9952649aba81e61d5d55290c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:00.014669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19643be9952649aba81e61d5d55290c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:00.015365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f55775ef-62ad-4b3b-a1fc-e461440c434f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19643be9952649aba81e61d5d55290c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:00.025832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1163/2319] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:10:59 2022] DELETE /identity/v3/users/19643be9952649aba81e61d5d55290c4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:00.683871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:00.685949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:00.709393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8cd32-4473-4d8d-a31c-1a179f7ecb9b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:00.826805 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:00.900623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:00.902880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:00.903259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:00.907374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] PATH_INFO: `/v3/projects/9ceec51e430e4a00b407068a2243be63` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:00.909638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ceec51e430e4a00b407068a2243be63)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:00.911909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:00.996057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ceec51e430e4a00b407068a2243be63', 'request_id': 'req-575d28b8-53b5-4450-9a81-9743763cb55d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:01.006169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ceec51e430e4a00b407068a2243be63', 'request_id': 'req-575d28b8-53b5-4450-9a81-9743763cb55d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:01.015486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ceec51e430e4a00b407068a2243be63', 'request_id': 'req-575d28b8-53b5-4450-9a81-9743763cb55d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:01.031420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] The token cache is being invalidate because project 9ceec51e430e4a00b407068a2243be63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:01.032281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:01.033580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-575d28b8-53b5-4450-9a81-9743763cb55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:01.036046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1157/2320] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:00 2022] DELETE /identity/v3/projects/9ceec51e430e4a00b407068a2243be63 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:01.041008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.041492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.041578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.076250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:01.077712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:01.156967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1164/2321] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:01.166662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.169233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.169595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.202301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:01.203671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:01.216769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1158/2322] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:01.221001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:01.233130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9e110-f244-45a4-aee8-31c7407af309 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:01.242132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:01.242762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.243043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.243316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.243970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:11:01.244467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:01.246284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:01.269935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:01.278658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1165/2323] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:11:01.281413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2abe3-bb3f-430d-af62-8cd21e1cb37c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:01.287529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:01.288497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.288497 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.288709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.289391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:01.291472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:01.370553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1159/2324] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:01 2022] POST /identity/v3/projects => generated 405 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:01.372181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:01.385539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5a87c-3a3f-4e32-aaba-61e892afede5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:01.493282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:01.494072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.494456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.494828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.495576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:01.497820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:01.531594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:01.538053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1166/2325] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:01 2022] POST /identity/v3/users => generated 303 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:11:01.558787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8abc-62ee-4778-b821-4007ebb99c8e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:01.571998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:01.573003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.573487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.574067 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.575370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:01.578189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:01.596247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1160/2326] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:11:01.599543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:01.617676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4269896c-24cb-4d92-9ec0-724208df3957 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:01.623662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0abee-f051-4d02-bf6b-7e9d28854d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:01.624436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0abee-f051-4d02-bf6b-7e9d28854d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.624780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0abee-f051-4d02-bf6b-7e9d28854d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.625122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0abee-f051-4d02-bf6b-7e9d28854d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.625829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0abee-f051-4d02-bf6b-7e9d28854d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:01.627894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0abee-f051-4d02-bf6b-7e9d28854d49 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:01.631185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:01.647021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1167/2327] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:01.648388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:01.660777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde46f61-e7c8-416f-b8a3-d0ed5b12bc6e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:01.667622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:01.668875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.669751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.669751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] PATH_INFO: `/v3/projects/c5e38d394f1842cc97385fd4a198c796/users/cdba84bcbdd84a8392f7b506b35cddfe/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.691391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5e38d394f1842cc97385fd4a198c796, user_id=cdba84bcbdd84a8392f7b506b35cddfe, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:01.695740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:01.728906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1161/2328] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:01 2022] PUT /identity/v3/projects/c5e38d394f1842cc97385fd4a198c796/users/cdba84bcbdd84a8392f7b506b35cddfe/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:01.742265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:01.742778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:01.743192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:01.787451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] MFA Rules not processed for user `cdba84bcbdd84a8392f7b506b35cddfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:01.790297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:01.907093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1168/2329] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:01 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:11:01.913287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.010062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57913c-4508-4043-9161-c1afc951fea2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.018846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.019550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.019836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.020138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.023167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.023167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.053298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:02.084863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.099282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1162/2330] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:01 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:02.111167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446b70e-c8a0-45c7-b174-76572bf4efbe None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.211388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.212366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.212655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.212971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.213825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.216555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.249505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1169/2331] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:02 2022] POST /identity/v3/users => generated 308 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:02.255049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.269937 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4dc65-92c8-49e0-947e-8762d525089b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.285508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.286309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.286729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.287314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.289936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.290987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.315610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1163/2332] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:02.316609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.331187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d71b5-fa78-47db-b91a-64a110d54c32 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.336979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.337806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.338183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.338586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] PATH_INFO: `/v3/projects/25de751cf2e44932bd57f3e8be93c910/users/a322a28465154387a950577f0b628bf5/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.364066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25de751cf2e44932bd57f3e8be93c910, user_id=a322a28465154387a950577f0b628bf5, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.366810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.403666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1170/2333] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:02 2022] PUT /identity/v3/projects/25de751cf2e44932bd57f3e8be93c910/users/a322a28465154387a950577f0b628bf5/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:02.407571 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.419116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52b91a-c482-480f-8e42-035763c5b1cd admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.539613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40bea-2465-4274-8fec-9c845558e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.540340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40bea-2465-4274-8fec-9c845558e3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.540614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40bea-2465-4274-8fec-9c845558e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.540893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40bea-2465-4274-8fec-9c845558e3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.541554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40bea-2465-4274-8fec-9c845558e3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.543253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40bea-2465-4274-8fec-9c845558e3f6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.555718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1164/2334] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:02.559974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.573048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79362000-f38b-447c-9f0e-c0d3fc42ea6a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.581112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.582080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.582481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.582854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] PATH_INFO: `/v3/projects/25de751cf2e44932bd57f3e8be93c910/users/a322a28465154387a950577f0b628bf5/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.584984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25de751cf2e44932bd57f3e8be93c910, user_id=a322a28465154387a950577f0b628bf5, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.587968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.601327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1171/2335] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:02 2022] PUT /identity/v3/projects/25de751cf2e44932bd57f3e8be93c910/users/a322a28465154387a950577f0b628bf5/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:02.617508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.617886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.618784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.660733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] MFA Rules not processed for user `a322a28465154387a950577f0b628bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:02.662161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:02.748760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1165/2336] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:02 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:02.757026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:02.759004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:02.824293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:02.840642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e3445-04d9-4706-90ee-c093c9fa4486 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:02.844942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47910ade-c111-4ee4-b14a-cacd3aad45eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:02.846199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910ade-c111-4ee4-b14a-cacd3aad45eb service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:02.846578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910ade-c111-4ee4-b14a-cacd3aad45eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:02.846955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47910ade-c111-4ee4-b14a-cacd3aad45eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:02.861077 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47910ade-c111-4ee4-b14a-cacd3aad45eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:02.863765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47910ade-c111-4ee4-b14a-cacd3aad45eb service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:02.888764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1172/2337] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:02 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:03.047958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:03.151734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704fcf66-7146-4cfa-b51c-4e3e597c859c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:03.159582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:03.160659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:03.160945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:03.161227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] PATH_INFO: `/v3/users/cdba84bcbdd84a8392f7b506b35cddfe` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:03.162557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdba84bcbdd84a8392f7b506b35cddfe)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:03.164424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:03.173885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:03.292575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdba84bcbdd84a8392f7b506b35cddfe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:03.293286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdba84bcbdd84a8392f7b506b35cddfe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:03.306074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdba84bcbdd84a8392f7b506b35cddfe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:03.325467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdba84bcbdd84a8392f7b506b35cddfe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:03.337180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdba84bcbdd84a8392f7b506b35cddfe'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:03.351341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1166/2338] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:03 2022] DELETE /identity/v3/users/cdba84bcbdd84a8392f7b506b35cddfe => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:03.355054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:03.355539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:03.355663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:03.406084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:03.408129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:03.573962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:03.578495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:03.674917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1173/2339] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:03.686733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:03.687483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:03.817688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9d674-84e8-4ab1-966c-bdd041460195 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:03.831385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:03.832518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:03.832518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:03.832796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:03.839194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:03.853527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75a126-c88f-4cd3-840b-501e57ef1267 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:03.865126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:03.865760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:03.866040 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:03.866309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:03.875460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:03.877215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:03.886920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:03.890217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:03.895748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1167/2340] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:11:03.906286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1174/2341] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:04.601093 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:04.602903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:04.642413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b99d4e-b395-42cc-8cbe-3e052280f1b0 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:04.651961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:04.652640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:04.653383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:04.653383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] PATH_INFO: `/v3/projects/c5e38d394f1842cc97385fd4a198c796` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:04.654539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5e38d394f1842cc97385fd4a198c796)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:04.656500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:04.717969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5e38d394f1842cc97385fd4a198c796', 'request_id': 'req-cfb41e36-f13c-48d7-963b-e056298c288b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.728729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5e38d394f1842cc97385fd4a198c796', 'request_id': 'req-cfb41e36-f13c-48d7-963b-e056298c288b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.742310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5e38d394f1842cc97385fd4a198c796', 'request_id': 'req-cfb41e36-f13c-48d7-963b-e056298c288b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.743601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:04.756212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] The token cache is being invalidate because project c5e38d394f1842cc97385fd4a198c796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:04.756474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.757282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-cfb41e36-f13c-48d7-963b-e056298c288b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.762558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1168/2342] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:04 2022] DELETE /identity/v3/projects/c5e38d394f1842cc97385fd4a198c796 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:04.766221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:04.776921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cf178-40d6-41a7-a0c2-74fff63e4798 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:04.850929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:04.851678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:04.851959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:04.852265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] PATH_INFO: `/v3/users/a322a28465154387a950577f0b628bf5` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:04.853457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a322a28465154387a950577f0b628bf5)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:04.855385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:04.886915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:04.911896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:04.912466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:04.912646 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:04.920314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a322a28465154387a950577f0b628bf5'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.928293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a322a28465154387a950577f0b628bf5'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.938146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a322a28465154387a950577f0b628bf5'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.946174 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] MFA Rules not processed for user `8a89943b6692461789bec2e224e845c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:04.948121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:04.957991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a322a28465154387a950577f0b628bf5'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.958877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a322a28465154387a950577f0b628bf5'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:04.970679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1175/2343] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:04 2022] DELETE /identity/v3/users/a322a28465154387a950577f0b628bf5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:05.074635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1169/2344] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:05.091196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:05.092892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:05.191459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6eeac3-f8c1-4f88-b94e-ff6d9c373246 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:05.198674 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:05.199578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.199879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.200287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.202460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:05.277339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:05.279186 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d60ae-0d54-4216-85f2-2ce5a8262a46 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:05.291010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1176/2345] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:11:05.320951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:05.322878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:05.347065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d69ab9-f0b0-485a-a77d-1b6242ea2265 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:05.449839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:05.450590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.450882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.451288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] PATH_INFO: `/v3/projects/25de751cf2e44932bd57f3e8be93c910` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.452761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=25de751cf2e44932bd57f3e8be93c910)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:05.455002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:05.525283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25de751cf2e44932bd57f3e8be93c910', 'request_id': 'req-dd4b8bc9-6008-4eac-b365-b922b6508fec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:05.540715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25de751cf2e44932bd57f3e8be93c910', 'request_id': 'req-dd4b8bc9-6008-4eac-b365-b922b6508fec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:05.555602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25de751cf2e44932bd57f3e8be93c910', 'request_id': 'req-dd4b8bc9-6008-4eac-b365-b922b6508fec'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:05.584627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] The token cache is being invalidate because project 25de751cf2e44932bd57f3e8be93c910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:05.584959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:05.585841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-dd4b8bc9-6008-4eac-b365-b922b6508fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:05.591614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1170/2346] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:05 2022] DELETE /identity/v3/projects/25de751cf2e44932bd57f3e8be93c910 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:05.593314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.593779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.594064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.618516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:05.619783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:05.665177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:05.693130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1177/2347] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:11:05.700982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.701299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.701538 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.724491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:05.725963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:05.737842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1171/2348] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:05.744346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:05.770154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298f6b2-775c-4da5-ba67-2f3aea6e955b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:05.795688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:05.799094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.799094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.799371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.800082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:11:05.800393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:05.802029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:05.815737 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1178/2349] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:11:05.820691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:05.826750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:05.836751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d03d63-5d44-4317-b81c-afde0103e86e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:05.843976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:05.844690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.845275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.845565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.846514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:05.848619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:05.886559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1172/2350] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:05 2022] POST /identity/v3/projects => generated 355 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:11:05.890597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:05.901275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097230c7-90f2-46a2-ace6-2df389e1352b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:05.977803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:05.978470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:05.978758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:05.979031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:05.979813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:05.982066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.010430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1179/2351] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:05 2022] POST /identity/v3/users => generated 278 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:06.014828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.027428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f45b9-6b15-45d3-9b86-6dd38128763d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.033872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.034505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.034774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.035309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.035965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.037695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.070310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1173/2352] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:06.073334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.085365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ed5d2-b070-4561-aa4c-b80af1dc6f65 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.091219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa7cd4-8a6c-43b3-a9c4-1fdb4283b0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.091847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa7cd4-8a6c-43b3-a9c4-1fdb4283b0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.092216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa7cd4-8a6c-43b3-a9c4-1fdb4283b0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.093058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa7cd4-8a6c-43b3-a9c4-1fdb4283b0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.093210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa7cd4-8a6c-43b3-a9c4-1fdb4283b0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.095645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa7cd4-8a6c-43b3-a9c4-1fdb4283b0d0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.117078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1180/2353] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:06.119389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.134269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aeb2dc-0a49-4e1f-9cac-293d062d0ed8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.140209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.141101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.141425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.141798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] PATH_INFO: `/v3/projects/c989794337704607886bcae4093162ae/users/1c16be6c0dad4ff4b34f5794cf19610d/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.163443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c989794337704607886bcae4093162ae, user_id=1c16be6c0dad4ff4b34f5794cf19610d, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.166965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.192211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1174/2354] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:06 2022] PUT /identity/v3/projects/c989794337704607886bcae4093162ae/users/1c16be6c0dad4ff4b34f5794cf19610d/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:06.203936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.204380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.204684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.259082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] MFA Rules not processed for user `1c16be6c0dad4ff4b34f5794cf19610d`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:06.260473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:06.355276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1181/2355] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:06.359835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.430696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604cbd-1377-489b-9e9a-c6f07d608aff admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.435926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.436615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.436867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.437144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.438190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.439826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.469647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1175/2356] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:06 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:11:06.474036 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.486656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bac611-66c1-4f43-98e1-b13e277bf107 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.530871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:06.569209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.570443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.571007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.571559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.572726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.576581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.603816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1182/2357] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:06 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:11:06.605193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.616797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acb739-d659-4fd5-991c-14616f41925c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.621987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.622591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.623025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.623332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.623953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.625619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.637869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1176/2358] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:06.643130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.655965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e07704-35b7-44b4-acf9-3214c5f1bc6a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.662009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.662683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.663028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.663315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] PATH_INFO: `/v3/projects/032a134ed5ff4d918927774696c412ed/users/84fdd85d75424789b381884a184dd5cf/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.680173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032a134ed5ff4d918927774696c412ed, user_id=84fdd85d75424789b381884a184dd5cf, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.683565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.706757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1183/2359] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:06 2022] PUT /identity/v3/projects/032a134ed5ff4d918927774696c412ed/users/84fdd85d75424789b381884a184dd5cf/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:06.708117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.719530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9a29-0fdd-43a1-8a18-2dc5040495f5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.807930 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8f0a8-1655-41fc-a8cd-e01626f7d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.808864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8f0a8-1655-41fc-a8cd-e01626f7d094 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.809146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8f0a8-1655-41fc-a8cd-e01626f7d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.809418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8f0a8-1655-41fc-a8cd-e01626f7d094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.810209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8f0a8-1655-41fc-a8cd-e01626f7d094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.818533 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8f0a8-1655-41fc-a8cd-e01626f7d094 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.835499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1177/2360] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:06.839989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:06.857792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077bf09-9f70-4d23-9d27-bedbb268bc01 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:06.864514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:06.865245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.865540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.866595 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] PATH_INFO: `/v3/projects/032a134ed5ff4d918927774696c412ed/users/84fdd85d75424789b381884a184dd5cf/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.868947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=032a134ed5ff4d918927774696c412ed, user_id=84fdd85d75424789b381884a184dd5cf, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:06.872033 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:06.895214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1184/2361] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:06 2022] PUT /identity/v3/projects/032a134ed5ff4d918927774696c412ed/users/84fdd85d75424789b381884a184dd5cf/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:06.897655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c292486-c357-4c20-b270-5f525e32779e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:06.897948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c292486-c357-4c20-b270-5f525e32779e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:06.898285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c292486-c357-4c20-b270-5f525e32779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:06.955056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3c292486-c357-4c20-b270-5f525e32779e None None] MFA Rules not processed for user `84fdd85d75424789b381884a184dd5cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:06.956584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3c292486-c357-4c20-b270-5f525e32779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:06.968966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:07.078002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1178/2362] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:07.091089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:07.094256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:07.196421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911ed3d9-5b13-461f-bf4e-13f968c3d3f7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:07.200667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11149f3d-4795-4f6f-8689-67919af8e828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:07.201430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11149f3d-4795-4f6f-8689-67919af8e828 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:07.201769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11149f3d-4795-4f6f-8689-67919af8e828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:07.202105 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11149f3d-4795-4f6f-8689-67919af8e828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:07.214517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11149f3d-4795-4f6f-8689-67919af8e828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:07.216406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11149f3d-4795-4f6f-8689-67919af8e828 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:07.229902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1185/2363] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:07.357677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c292486-c357-4c20-b270-5f525e32779e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:07.372890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c292486-c357-4c20-b270-5f525e32779e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:07.381561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:07.382209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:07.382660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:07.383091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:07.506614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:07.509150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:07.524496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1179/2364] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:08.124779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:08.124779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:08.124779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:08.151914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:08.153447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:08.262872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1186/2365] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:11:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:08.278255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:08.280358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:08.369743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47f9c8-f41e-412d-aee6-59f56e03ad22 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:08.374776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9880a-23f8-4f55-83ba-9e86f63c9e04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:08.375391 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9880a-23f8-4f55-83ba-9e86f63c9e04 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:08.375675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9880a-23f8-4f55-83ba-9e86f63c9e04 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:08.376635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9880a-23f8-4f55-83ba-9e86f63c9e04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:08.389841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9880a-23f8-4f55-83ba-9e86f63c9e04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:08.392203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9880a-23f8-4f55-83ba-9e86f63c9e04 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:08.402508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1180/2366] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:11:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:11:18.233874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:18.398059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:18.968257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:18.970545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:19.099312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ba144-5950-43d3-9df2-d4db9817f5b4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:19.107578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:19.108391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:19.108677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:19.108957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:19.112034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:19.129902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:19.131596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:19.160098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1187/2367] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:23.772398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:23.773299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:23.773601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:23.800778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] MFA Rules not processed for user `16696c34de254f3babf538c879f2ce49`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:23.801993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:23.865597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1181/2368] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:24.209877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:24.211587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:24.291873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce98fcd-a726-4f98-b2ad-a86917731dd3 service nova] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:24.295887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:24.296540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:24.297708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:24.297961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:24.299996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:24.312383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:24.314041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a0d2d-176e-46f7-a106-02d25422ca7a service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:24.327101 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1188/2369] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:24.520215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:24.621276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7ae1a-861d-4857-9078-e8a6b6b8354a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:24.637130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:24.637991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:24.638380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:24.638752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] PATH_INFO: `/v3/users/1c16be6c0dad4ff4b34f5794cf19610d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:24.640116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c16be6c0dad4ff4b34f5794cf19610d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:24.643294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:24.707269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c16be6c0dad4ff4b34f5794cf19610d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:24.707965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c16be6c0dad4ff4b34f5794cf19610d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:24.717500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c16be6c0dad4ff4b34f5794cf19610d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:24.738782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c16be6c0dad4ff4b34f5794cf19610d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:24.746877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c16be6c0dad4ff4b34f5794cf19610d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:24.754111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1182/2370] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:24 2022] DELETE /identity/v3/users/1c16be6c0dad4ff4b34f5794cf19610d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:24.758378 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:24.758668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:24.758997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:24.780145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:24.782633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:24.868379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1189/2371] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:11:24.883360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:24.884968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:24.929386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:24.987957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f435a2e-60a0-4721-83fe-a77808b07634 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:24.993369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:24.994629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:24.994629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:24.994629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:25.006256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:25.007620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:25.038328 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1183/2372] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:25.539667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:25.541779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:25.558626 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a21e9-41ab-4c3d-bd1b-75b7779af69a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:25.569729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:25.571067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:25.571399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:25.571675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] PATH_INFO: `/v3/projects/c989794337704607886bcae4093162ae` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:25.573514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c989794337704607886bcae4093162ae)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:25.576852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:25.611111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:25.643050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c989794337704607886bcae4093162ae', 'request_id': 'req-b9edce8d-f25f-489a-860f-2753b232a953'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.651994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c989794337704607886bcae4093162ae', 'request_id': 'req-b9edce8d-f25f-489a-860f-2753b232a953'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.660074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c989794337704607886bcae4093162ae', 'request_id': 'req-b9edce8d-f25f-489a-860f-2753b232a953'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.681490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] The token cache is being invalidate because project c989794337704607886bcae4093162ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:25.681797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.682735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9edce8d-f25f-489a-860f-2753b232a953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.684675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1190/2373] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:25 2022] DELETE /identity/v3/projects/c989794337704607886bcae4093162ae => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:25.689092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:25.699704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae78c1dd-256f-41e6-9b3d-318656d98f3d service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:25.795861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:25.796630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:25.797891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:25.798171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] PATH_INFO: `/v3/users/84fdd85d75424789b381884a184dd5cf` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:25.799581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84fdd85d75424789b381884a184dd5cf)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:25.801648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:25.866366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84fdd85d75424789b381884a184dd5cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.867518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84fdd85d75424789b381884a184dd5cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.879074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84fdd85d75424789b381884a184dd5cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.896395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84fdd85d75424789b381884a184dd5cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.905962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84fdd85d75424789b381884a184dd5cf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:25.913781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1184/2374] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:25 2022] DELETE /identity/v3/users/84fdd85d75424789b381884a184dd5cf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:26.242274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87c76-8f4b-4533-91ab-dbcbb23c7bad None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:26.243473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87c76-8f4b-4533-91ab-dbcbb23c7bad None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:26.244437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd87c76-8f4b-4533-91ab-dbcbb23c7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:26.269017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8fd87c76-8f4b-4533-91ab-dbcbb23c7bad None None] MFA Rules not processed for user `81d937b8a6c842819458b6e3c23f80fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:26.270302 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8fd87c76-8f4b-4533-91ab-dbcbb23c7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:26.364324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1191/2375] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:26.401445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:26.402868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:26.483037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:26.521042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d60665-28fe-4545-a7a5-de5f134bc230 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:26.526557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:26.527423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:26.527864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:26.528324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] PATH_INFO: `/v3/projects/032a134ed5ff4d918927774696c412ed` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:26.529509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=032a134ed5ff4d918927774696c412ed)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:26.531835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:26.598977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '032a134ed5ff4d918927774696c412ed', 'request_id': 'req-2f0d816e-108f-4c0d-b5d3-3561390a9610'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:26.611146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '032a134ed5ff4d918927774696c412ed', 'request_id': 'req-2f0d816e-108f-4c0d-b5d3-3561390a9610'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:26.619080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '032a134ed5ff4d918927774696c412ed', 'request_id': 'req-2f0d816e-108f-4c0d-b5d3-3561390a9610'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:26.634682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] The token cache is being invalidate because project 032a134ed5ff4d918927774696c412ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:26.635012 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:26.635910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-2f0d816e-108f-4c0d-b5d3-3561390a9610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:26.638802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1185/2376] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:26 2022] DELETE /identity/v3/projects/032a134ed5ff4d918927774696c412ed => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:26.644548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:26.645394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:26.645850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:26.676946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:26.678156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:26.806811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1192/2377] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:11:26.815790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:26.817688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:26.817688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:26.852095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:26.853734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:26.865835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1186/2378] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:26.870888 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:26.886822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e50f8-247c-41c8-94e1-849429c8fcc3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:26.893043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:26.893681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:26.893958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:26.894354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:26.894876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:11:26.895615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:26.897563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:26.920431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1193/2379] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:11:26.923125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:26.947479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e8d063-33df-43a7-8952-70692418d51a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:26.957570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:26.959729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:26.960687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:26.960964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:26.962794 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:26.966945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.008833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1187/2380] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:26 2022] POST /identity/v3/projects => generated 365 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:27.009299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.021256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f90d48-3ca9-4ce2-a147-a9ef6b295ac8 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:27.083099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:27.126339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:27.127374 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.127700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.128151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.129742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:27.132415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.193886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1194/2381] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:27 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:11:27.195285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.211638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160c2ca1-1d76-4639-977a-748eee62392c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:27.218437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:27.219823 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.220145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.220422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.221064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:27.222656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.235076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1188/2382] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:27.240605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.266138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5e2b3-96e7-4a3e-a926-9dca83f70f73 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:27.292331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02059555-6f50-403f-bea4-5ed8b61afd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:27.292331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02059555-6f50-403f-bea4-5ed8b61afd8b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.292331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02059555-6f50-403f-bea4-5ed8b61afd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.292642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02059555-6f50-403f-bea4-5ed8b61afd8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.298970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02059555-6f50-403f-bea4-5ed8b61afd8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:27.301622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02059555-6f50-403f-bea4-5ed8b61afd8b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.336519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1195/2383] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:27.349160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.362861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616ace-29c6-4a17-922f-318b82ea67ef admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:27.379608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:27.380354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.380750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.381033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] PATH_INFO: `/v3/projects/c303e19bf2e0480ba70ab65703692170/users/c10b26dc1571443aa4f064e8e9a2c711/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.400458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c303e19bf2e0480ba70ab65703692170, user_id=c10b26dc1571443aa4f064e8e9a2c711, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:27.403751 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.425502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1189/2384] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:27 2022] PUT /identity/v3/projects/c303e19bf2e0480ba70ab65703692170/users/c10b26dc1571443aa4f064e8e9a2c711/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:27.438021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922c0ee-c4f9-4d8b-9d8e-8b5ae40c2b66 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.438295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922c0ee-c4f9-4d8b-9d8e-8b5ae40c2b66 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.438609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922c0ee-c4f9-4d8b-9d8e-8b5ae40c2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.473902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c922c0ee-c4f9-4d8b-9d8e-8b5ae40c2b66 None None] MFA Rules not processed for user `c10b26dc1571443aa4f064e8e9a2c711`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:27.475280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c922c0ee-c4f9-4d8b-9d8e-8b5ae40c2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:27.613699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1196/2385] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:27.615729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.668957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac65a2e-f8ad-4510-a9e6-b84169ff2382 None None] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.669517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac65a2e-f8ad-4510-a9e6-b84169ff2382 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.669517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac65a2e-f8ad-4510-a9e6-b84169ff2382 None None] PATH_INFO: `/` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.670905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1197/2386] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:11:27.679277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.679558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.680098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.726333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:27.730347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:27.764169 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8cd082-4d48-45a3-9204-9aa99d506e67 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:27.782035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:27.782668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.783040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.783212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.786152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:27.790434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.841382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:27.855072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1190/2387] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:27 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:27.862374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.876197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1198/2388] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:11:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:27.886748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:27.893029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:27.961499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7ff54-c815-48a0-85e5-edda5f080618 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:27.967798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:27.969307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:27.969434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:27.969603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:27.970150 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:27.970961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:27.973118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:27.985406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ea5e6-0652-4085-8f4a-e6af5ae7ce3c None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.015837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1191/2389] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:27 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:28.020141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.034354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4007ac6f-2458-4466-8f8d-ccb472aa0776 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.039643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.040520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.040824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.041107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.042134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.044097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.056372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1192/2390] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:11:28.058376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.058988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.059304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.059585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.060087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.074735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7af9f-18d4-411e-9fca-b32d042de818 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.080365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.081032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.081436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.081721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] PATH_INFO: `/v3/projects/642933ef91da494da2c4ea988658f97f/users/03ba3fc75b6e426588eac649e1396834/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.097360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642933ef91da494da2c4ea988658f97f, user_id=03ba3fc75b6e426588eac649e1396834, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.100306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.115373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1193/2391] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:28 2022] PUT /identity/v3/projects/642933ef91da494da2c4ea988658f97f/users/03ba3fc75b6e426588eac649e1396834/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:28.119458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.138009 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.139521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.151122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1199/2392] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:28.191305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a05e62-af4a-460f-b4d1-4b72e31cb0e3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.198438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc2740-b1c5-4774-9547-8a852f719ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.199117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2740-b1c5-4774-9547-8a852f719ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.199821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2740-b1c5-4774-9547-8a852f719ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.199821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc2740-b1c5-4774-9547-8a852f719ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.200479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc2740-b1c5-4774-9547-8a852f719ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.202124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cc2740-b1c5-4774-9547-8a852f719ed0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.215679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1194/2393] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:11:28.222025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.236405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54104ee1-14e6-4452-b00a-e5c4e414fd12 service glance] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.244959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee6c0b0-2261-47df-9333-3ee44c87e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.245625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6c0b0-2261-47df-9333-3ee44c87e302 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.245907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6c0b0-2261-47df-9333-3ee44c87e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.246225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee6c0b0-2261-47df-9333-3ee44c87e302 admin admin] PATH_INFO: `/v3/projects/642933ef91da494da2c4ea988658f97f/users/03ba3fc75b6e426588eac649e1396834/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.248716 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6c0b0-2261-47df-9333-3ee44c87e302 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642933ef91da494da2c4ea988658f97f, user_id=03ba3fc75b6e426588eac649e1396834, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.251749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee6c0b0-2261-47df-9333-3ee44c87e302 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.258351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e27882-f93a-449a-aee0-c6f5bde29f7f None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.259055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e27882-f93a-449a-aee0-c6f5bde29f7f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.259845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e27882-f93a-449a-aee0-c6f5bde29f7f None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.262105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1195/2394] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:11:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 11:11:28.264915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1200/2395] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:28 2022] PUT /identity/v3/projects/642933ef91da494da2c4ea988658f97f/users/03ba3fc75b6e426588eac649e1396834/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:28.274510 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.275106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.275545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.276021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.276322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.276604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.305188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] MFA Rules not processed for user `49f7536740a54bdbba5b6c81f732c439`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:28.306727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:28.313750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] MFA Rules not processed for user `03ba3fc75b6e426588eac649e1396834`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:28.315779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:28.364339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:28.378386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1196/2396] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:28.392955 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.395262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:28.412763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b0d21-3e34-4dbe-9247-62139673af04 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.417440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f51b3-fed2-4bd8-9485-3be7b9184667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.418440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f51b3-fed2-4bd8-9485-3be7b9184667 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.418905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f51b3-fed2-4bd8-9485-3be7b9184667 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.420189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f51b3-fed2-4bd8-9485-3be7b9184667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.424899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1201/2397] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:28.430627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f51b3-fed2-4bd8-9485-3be7b9184667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.432439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f51b3-fed2-4bd8-9485-3be7b9184667 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.434082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.435728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:28.443431 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1197/2398] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:28.464053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:28.534550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504c4e1-8af2-435f-a763-c744960942b5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.554258 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.554895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.555170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.555442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.567011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.568694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.579203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6251c7-34ae-43fc-9e86-d1d07220adcc None None] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.579743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6251c7-34ae-43fc-9e86-d1d07220adcc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.580293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6251c7-34ae-43fc-9e86-d1d07220adcc None None] PATH_INFO: `/` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.582192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1198/2399] 10.222.0.20 () {54 vars in 921 bytes} [Wed Aug 31 11:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 11:11:28.585239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1202/2400] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:28.594255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.594540 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.594810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.621422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] MFA Rules not processed for user `508f557ac5e34c67a31e4d55f8142a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:28.622748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:28.684167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.694567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd668034-ca5e-400f-9e43-d11ee2e6f1c5 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.699070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.699827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.700162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.700575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.721556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1199/2401] 10.222.0.20 () {58 vars in 1018 bytes} [Wed Aug 31 11:11:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:28.734427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:28.747504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33717f41-9291-4f14-a75e-5c1b79cec8b6 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:28.752856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12afc8-5916-454d-8fcf-a1ecc406155d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:28.753559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12afc8-5916-454d-8fcf-a1ecc406155d service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:28.753848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12afc8-5916-454d-8fcf-a1ecc406155d service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:28.754123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12afc8-5916-454d-8fcf-a1ecc406155d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:28.770967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12afc8-5916-454d-8fcf-a1ecc406155d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.774579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12afc8-5916-454d-8fcf-a1ecc406155d service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.778453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:28.783315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:28.792657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1200/2402] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:28.802418 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1203/2403] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:11:32.378015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:32.378522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:32.378606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:32.400305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:32.401418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:32.466018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1201/2404] 10.222.0.20 () {58 vars in 1009 bytes} [Wed Aug 31 11:11:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:32.494490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:32.495972 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:32.586700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f6f54-019e-4545-98f0-a4fd7892063c service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:32.591776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f59d5-1001-4884-84f5-7dc8f9d9cf7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:32.592436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f59d5-1001-4884-84f5-7dc8f9d9cf7b service swift] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:32.592719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f59d5-1001-4884-84f5-7dc8f9d9cf7b service swift] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:32.592997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f59d5-1001-4884-84f5-7dc8f9d9cf7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:32.601391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f59d5-1001-4884-84f5-7dc8f9d9cf7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:32.603060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f59d5-1001-4884-84f5-7dc8f9d9cf7b service swift] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:32.611127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1204/2405] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:11:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 11:11:38.683783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:38.769465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa21beb5-bf44-4bfb-ab08-1627b8278cd0 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:38.777611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:38.779051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:38.779220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:38.779461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] PATH_INFO: `/v3/users/c10b26dc1571443aa4f064e8e9a2c711` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:38.782380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10b26dc1571443aa4f064e8e9a2c711)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:38.787604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:38.801635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:38.865996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10b26dc1571443aa4f064e8e9a2c711'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:38.866785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10b26dc1571443aa4f064e8e9a2c711'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:38.880377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10b26dc1571443aa4f064e8e9a2c711'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:38.896357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10b26dc1571443aa4f064e8e9a2c711'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:38.907052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10b26dc1571443aa4f064e8e9a2c711'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:38.933057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1202/2406] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:38 2022] DELETE /identity/v3/users/c10b26dc1571443aa4f064e8e9a2c711 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:38.939038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:38.939569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:38.939929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:38.983929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:38.986067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:39.043623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:39.065980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1205/2407] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:39.075682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:39.077413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:39.228114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca13d5-eb87-4d49-b23d-34101ff64e16 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:39.236435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:39.237193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:39.237340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:39.237616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:39.253583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:39.255401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:39.269670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1203/2408] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:39.953662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:39.955462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:39.990879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854384d-c02e-406b-81ab-1b3a5b90dcac None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:39.996427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:39.997198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:39.997591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:39.997929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] PATH_INFO: `/v3/projects/c303e19bf2e0480ba70ab65703692170` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:39.999238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c303e19bf2e0480ba70ab65703692170)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:40.001938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:40.074241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c303e19bf2e0480ba70ab65703692170', 'request_id': 'req-af579d97-401f-486b-8f86-e0f2bdff41f4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.086631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c303e19bf2e0480ba70ab65703692170', 'request_id': 'req-af579d97-401f-486b-8f86-e0f2bdff41f4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.094615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c303e19bf2e0480ba70ab65703692170', 'request_id': 'req-af579d97-401f-486b-8f86-e0f2bdff41f4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.119724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] The token cache is being invalidate because project c303e19bf2e0480ba70ab65703692170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:40.120158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.120876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.122565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1206/2409] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:39 2022] DELETE /identity/v3/projects/c303e19bf2e0480ba70ab65703692170 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:40.127445 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:40.138487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75c2b2-eb80-4314-a028-3103acf8e236 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:40.209222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:40.209878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:40.210228 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:40.211178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] PATH_INFO: `/v3/users/03ba3fc75b6e426588eac649e1396834` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:40.212482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03ba3fc75b6e426588eac649e1396834)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:40.214529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:40.274886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:40.293956 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03ba3fc75b6e426588eac649e1396834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.294983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03ba3fc75b6e426588eac649e1396834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.305694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03ba3fc75b6e426588eac649e1396834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.339676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03ba3fc75b6e426588eac649e1396834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.349144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c15e9918-430f-4a91-9ca6-981bd988c420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03ba3fc75b6e426588eac649e1396834'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:40.360049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1204/2410] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:40 2022] DELETE /identity/v3/users/03ba3fc75b6e426588eac649e1396834 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:40.824535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:40.826060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:40.856121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af579d97-401f-486b-8f86-e0f2bdff41f4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:40.945678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:40.946396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:40.946745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:40.947088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] PATH_INFO: `/v3/projects/642933ef91da494da2c4ea988658f97f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:40.948439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=642933ef91da494da2c4ea988658f97f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:40.964463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.031058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642933ef91da494da2c4ea988658f97f', 'request_id': 'req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:41.039082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642933ef91da494da2c4ea988658f97f', 'request_id': 'req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:41.053184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642933ef91da494da2c4ea988658f97f', 'request_id': 'req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:41.072596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:41.082608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] The token cache is being invalidate because project 642933ef91da494da2c4ea988658f97f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:41.083344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:41.084345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-2fabfbf0-c1dc-4915-85d8-c695cffa9b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:41.086485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1207/2411] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:40 2022] DELETE /identity/v3/projects/642933ef91da494da2c4ea988658f97f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:41.095755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.095755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.095755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.114567 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:41.115941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:41.200615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1205/2412] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:41.208497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.209012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.209415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.241491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:41.242721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:41.278547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1208/2413] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:41.283861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.298598 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd9262-8c48-4ff4-8777-66e8af5a5666 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.307542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.308220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.308504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.308779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.309441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:11:41.309839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.311658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.329841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1206/2414] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:11:41.334811 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.349985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbaed8-08d1-44f1-9c81-002abb3f9718 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.361119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.361919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.362192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.362470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.363117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.365707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.408748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1209/2415] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:41 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:41.413126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.427986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f1371-59d8-4504-b80e-c4073ae740db admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.520941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.521722 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.522006 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.522279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.522927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.524916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.547921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1207/2416] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:41 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:41.552256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.564268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ec888-ac55-401a-8c20-32ba563eccd4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.569666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.570279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.570561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.570833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.571480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.573121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.585106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1210/2417] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:41.589251 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.608995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e833d-68e6-4782-87f9-c82902f05237 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.615226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbbdd4e-1a25-4a94-bfcf-64fc4f59e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.615850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbdd4e-1a25-4a94-bfcf-64fc4f59e51c admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.616164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbdd4e-1a25-4a94-bfcf-64fc4f59e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.616435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbbdd4e-1a25-4a94-bfcf-64fc4f59e51c admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.617119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbdd4e-1a25-4a94-bfcf-64fc4f59e51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.618886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbbdd4e-1a25-4a94-bfcf-64fc4f59e51c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.633597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1208/2418] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:41.637376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.649351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e97c43-c8f7-4d45-b033-b402264c5ded admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.654141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.654803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.655111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.655355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] PATH_INFO: `/v3/projects/ad83b7406701434c9986f0beabead3a4/users/edf1bb2b59cc4069978873a9bd214940/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.672140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad83b7406701434c9986f0beabead3a4, user_id=edf1bb2b59cc4069978873a9bd214940, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.674695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.689554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1211/2419] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:41 2022] PUT /identity/v3/projects/ad83b7406701434c9986f0beabead3a4/users/edf1bb2b59cc4069978873a9bd214940/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:41.701078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.701359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.701634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.732196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] MFA Rules not processed for user `edf1bb2b59cc4069978873a9bd214940`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:41.733333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:41.737731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:41.834610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1209/2420] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:41.847710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:41.849498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:41.944270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cde967-40f8-4230-9f57-522a9a729908 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:41.948484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737dc8-15b4-479e-8c85-a3734444ab35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:41.949244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737dc8-15b4-479e-8c85-a3734444ab35 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:41.949534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737dc8-15b4-479e-8c85-a3734444ab35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:41.950313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737dc8-15b4-479e-8c85-a3734444ab35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:41.961494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737dc8-15b4-479e-8c85-a3734444ab35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:41.963123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737dc8-15b4-479e-8c85-a3734444ab35 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:41.973670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1212/2421] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:41.999680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:42.080258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9cbcc-2fe6-4ffe-b9c2-9673e0df72aa None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:42.086078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:42.086777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:42.087059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:42.087421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] PATH_INFO: `/v3/users/edf1bb2b59cc4069978873a9bd214940` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:42.088581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edf1bb2b59cc4069978873a9bd214940)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:42.090672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:42.167261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edf1bb2b59cc4069978873a9bd214940'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.168580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edf1bb2b59cc4069978873a9bd214940'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.191374 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edf1bb2b59cc4069978873a9bd214940'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.211448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edf1bb2b59cc4069978873a9bd214940'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.228217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edf1bb2b59cc4069978873a9bd214940'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.237732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1210/2422] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:11:41 2022] DELETE /identity/v3/users/edf1bb2b59cc4069978873a9bd214940 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:42.244941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:42.245311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:42.245665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:42.277688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:42.279064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:42.364474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1213/2423] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:42.376513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:42.377971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:42.412429 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:42.448636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cbb05-26c4-4551-83bd-fc951cb5ffe0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:42.452332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e5023-c36f-4e5d-8164-1d60d3ed57fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:42.453802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e5023-c36f-4e5d-8164-1d60d3ed57fe service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:42.454200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e5023-c36f-4e5d-8164-1d60d3ed57fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:42.454386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e5023-c36f-4e5d-8164-1d60d3ed57fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:42.464959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e5023-c36f-4e5d-8164-1d60d3ed57fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:42.466473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e5023-c36f-4e5d-8164-1d60d3ed57fe service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:42.477034 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1211/2424] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:42.831960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:42.834909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:42.847053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6b9cc-2ee7-4ad8-b4f5-5ed615126996 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:42.854089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:42.854825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:42.855212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:42.855544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] PATH_INFO: `/v3/projects/ad83b7406701434c9986f0beabead3a4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:42.856884 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad83b7406701434c9986f0beabead3a4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:42.866992 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:42.905137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:42.918408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad83b7406701434c9986f0beabead3a4', 'request_id': 'req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.924872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad83b7406701434c9986f0beabead3a4', 'request_id': 'req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.930438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad83b7406701434c9986f0beabead3a4', 'request_id': 'req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.942917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] The token cache is being invalidate because project ad83b7406701434c9986f0beabead3a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:11:42.943205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.943823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-9b5b1e7f-f58a-4bee-81e6-2b92a81c3a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:11:42.945338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1214/2425] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:11:42 2022] DELETE /identity/v3/projects/ad83b7406701434c9986f0beabead3a4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:11:42.950965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:42.951248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:42.951523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:42.976123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:42.977348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:43.041437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1212/2426] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:43.049935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.050481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.050481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.070945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:43.072205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:43.081432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1215/2427] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:43.085719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.096390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f08f28-3137-420b-a1c2-ac90f3d1e82f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.106669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.107299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.107579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.108027 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.108556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:11:43.108886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.111107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.123118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1213/2428] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:11:43.127765 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.137000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888692d4-355d-4ad1-b6d4-957238d3e2fc None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.145458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.146085 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.146446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.146726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.147379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.149362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.180644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1216/2429] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:11:43 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:11:43.184927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.195754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c698b7-7898-42cb-971e-c930656d19e3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.262265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.263028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.263619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.263795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.264505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.267898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.299181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1214/2430] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:11:43 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:11:43.304064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.316089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c3cea4-7408-4bbd-9025-0988860d297a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.320968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.321581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.321859 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.322193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.322840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.324675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.336907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1217/2431] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:11:43.340836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.351311 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eef3c9-4e7d-4a7e-9b44-0f08bebcfbf8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.356116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f814ed5-cee3-4603-b899-8b821af752a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.356801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f814ed5-cee3-4603-b899-8b821af752a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.357138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f814ed5-cee3-4603-b899-8b821af752a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.357476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f814ed5-cee3-4603-b899-8b821af752a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.358192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f814ed5-cee3-4603-b899-8b821af752a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.361465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f814ed5-cee3-4603-b899-8b821af752a4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.373460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1215/2432] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:11:43.378608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.389631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8e1bf-05f1-4c15-b7d3-be50804c2240 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.394539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.395222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.395505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.395781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] PATH_INFO: `/v3/projects/8837c791843f49339e74e9bc5035f617/users/b14c5d9d54064be88abdbd0b9d8b24dd/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.412632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8837c791843f49339e74e9bc5035f617, user_id=b14c5d9d54064be88abdbd0b9d8b24dd, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.415267 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.430329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1218/2433] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:11:43 2022] PUT /identity/v3/projects/8837c791843f49339e74e9bc5035f617/users/b14c5d9d54064be88abdbd0b9d8b24dd/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:11:43.442319 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.442839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.443296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.475546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] MFA Rules not processed for user `b14c5d9d54064be88abdbd0b9d8b24dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:43.476981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:43.500427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:43.571672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1216/2434] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:11:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:43.582400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:43.584485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:43.647264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146af239-29a3-4493-b615-001cbb55749b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:43.651025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68c883-9a4c-4342-9c2b-b227333115cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:43.651624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68c883-9a4c-4342-9c2b-b227333115cc service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:43.651902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68c883-9a4c-4342-9c2b-b227333115cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:43.652204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68c883-9a4c-4342-9c2b-b227333115cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:43.662298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68c883-9a4c-4342-9c2b-b227333115cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:43.665415 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68c883-9a4c-4342-9c2b-b227333115cc service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:43.675693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1219/2435] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:48.943317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:48.944757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:49.003931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344404e6-34ae-4cc4-904f-0ebf55a9a634 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:49.007618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:49.008259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:49.008544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:49.008827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:49.010580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:49.075635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:49.077138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:49.088089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1217/2436] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:11:50.000185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:50.000764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:50.000764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:50.020333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] MFA Rules not processed for user `938980e29ada4c4eba2a4172845396e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:11:50.021541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:50.080082 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1220/2437] 10.222.0.20 () {58 vars in 967 bytes} [Wed Aug 31 11:11:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:11:50.102884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:50.104370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:50.182346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091633b-a833-409c-8bf9-fe05c9b4f4db service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:50.188396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '10831ea9480d488ab2a81bece14ff090', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:50.188959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:50.189305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:50.189647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:50.201813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:50.204056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:50.214070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1218/2438] 10.222.0.20 () {60 vars in 1384 bytes} [Wed Aug 31 11:11:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 11:11:56.590483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:11:56.592196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:56.602069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:11:56.705205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb3b46-89dd-4faa-a170-4a9a5c701e9f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:11:56.713068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:11:56.713842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:11:56.714225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:11:56.714723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:11:56.717271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:11:56.731370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:11:56.733815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:11:56.745386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1221/2439] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:11:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:00.209736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:04.818662 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:04.828431 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103b842-2bc8-4c0c-ba83-2847cf9d8ef0 service swift] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:04.833621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:04.834331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:04.834676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:04.835048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:04.836920 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:04.900644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:04.902202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:04.914325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1219/2440] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:06.741029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=32 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:08.151797 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:08.154820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:08.225558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:08.227229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:08.245388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0630517-507d-4768-a2b5-d64885f6e650 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:08.254122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:08.255143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.255603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.256131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] PATH_INFO: `/v3/users/d0e5dae087184c56b1cb72fb7705d4d5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.260025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e5dae087184c56b1cb72fb7705d4d5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:08.263389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:08.270615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c0f5a-1bc5-4bb4-aff7-ee4f291fdd04 service nova] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:08.283245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:08.283901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.284212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.284512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.286564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:08.300347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:08.302719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:08.315722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1222/2441] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:08.337692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e5dae087184c56b1cb72fb7705d4d5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.338101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e5dae087184c56b1cb72fb7705d4d5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.347040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e5dae087184c56b1cb72fb7705d4d5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.362590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e5dae087184c56b1cb72fb7705d4d5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.369428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-796b2db9-dbde-4674-8b54-78b80ebb7e0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e5dae087184c56b1cb72fb7705d4d5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.375378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1220/2442] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:08 2022] DELETE /identity/v3/users/d0e5dae087184c56b1cb72fb7705d4d5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:08.558861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:08.570207 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f9ab9-2fde-41c1-9fe3-333734a76964 service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:08.631005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:08.631627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.631906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.632210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] PATH_INFO: `/v3/projects/173d9af664764ba5bd694dab38d8bb6a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.633327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=173d9af664764ba5bd694dab38d8bb6a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:08.635211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:08.687762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173d9af664764ba5bd694dab38d8bb6a', 'request_id': 'req-75280bed-13bd-42b4-aae4-0f701dd484dd'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.694741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173d9af664764ba5bd694dab38d8bb6a', 'request_id': 'req-75280bed-13bd-42b4-aae4-0f701dd484dd'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.700459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173d9af664764ba5bd694dab38d8bb6a', 'request_id': 'req-75280bed-13bd-42b4-aae4-0f701dd484dd'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.713779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] The token cache is being invalidate because project 173d9af664764ba5bd694dab38d8bb6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:08.714072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.714690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-75280bed-13bd-42b4-aae4-0f701dd484dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:08.716219 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1223/2443] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:08 2022] DELETE /identity/v3/projects/173d9af664764ba5bd694dab38d8bb6a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:08.722100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.722386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.722660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.742369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:08.743559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:08.807901 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1221/2444] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:08.817003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.817281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.817552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.840109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:08.841619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:08.850606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1224/2445] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:08.855019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:08.865506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a2679-33fe-4da0-841a-8fc0d301eff3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:08.870119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:08.870760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.871063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.871344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.871996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:08.872322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:08.873925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:08.875437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:08.889202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1222/2446] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:12:08.894289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:08.904643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990eabe-1eaf-49d1-bbde-7d5163d3c4db None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:08.915808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:08.916790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:08.917065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:08.917334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:08.918056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:08.920449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:08.932844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:08.949443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1225/2447] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:08 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:08.954233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:08.963996 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ab0d1-3846-408f-8b6b-ae9e40048b83 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.024383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.024995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.025277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.025559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.026189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.028220 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.050033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1223/2448] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:08 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:09.054220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:09.068997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea121d-3e7e-4e95-9997-a900eb0b650a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.073952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.074736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.074885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.075157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.075805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.077470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.088945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1226/2449] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:09.093079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:09.103635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdcc95-2703-4997-a9ee-35698cc44d2e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.108542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3685e2-8de2-4f04-97c0-3a6342c3dca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.109168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3685e2-8de2-4f04-97c0-3a6342c3dca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.109443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3685e2-8de2-4f04-97c0-3a6342c3dca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.109767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3685e2-8de2-4f04-97c0-3a6342c3dca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.110409 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3685e2-8de2-4f04-97c0-3a6342c3dca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.111998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3685e2-8de2-4f04-97c0-3a6342c3dca1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.122834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1224/2450] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:09.127760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:09.137830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d51c-cd37-4bf8-ad2c-169c8d2fe213 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.144168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.144843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.145185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.145561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] PATH_INFO: `/v3/projects/57a726146b784308aadcbf92eb524483/users/87a0a7593e5c49cea2957f574a266a29/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.162873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=57a726146b784308aadcbf92eb524483, user_id=87a0a7593e5c49cea2957f574a266a29, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.165557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.179690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1227/2451] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:09 2022] PUT /identity/v3/projects/57a726146b784308aadcbf92eb524483/users/87a0a7593e5c49cea2957f574a266a29/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:09.189925 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.190241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.190518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.222893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] MFA Rules not processed for user `87a0a7593e5c49cea2957f574a266a29`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:09.224242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:09.290066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1225/2452] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:09.304254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:09.305933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:09.382551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0a4e-86ca-43cf-9232-89c76cd43d0f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.388647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d7e4d-19eb-4ac2-8077-683d57d760ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.389265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d7e4d-19eb-4ac2-8077-683d57d760ae service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.389664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d7e4d-19eb-4ac2-8077-683d57d760ae service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.389946 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d7e4d-19eb-4ac2-8077-683d57d760ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.408035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d7e4d-19eb-4ac2-8077-683d57d760ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.408035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4d7e4d-19eb-4ac2-8077-683d57d760ae service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.412089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1228/2453] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:09.646894 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:09.710506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5955fcf-b97d-4190-879e-d350c8477b82 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.715115 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.715740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.716046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.716322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] PATH_INFO: `/v3/users/87a0a7593e5c49cea2957f574a266a29` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.717440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a0a7593e5c49cea2957f574a266a29)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.719943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.722398 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:09.768680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a0a7593e5c49cea2957f574a266a29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:09.769459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a0a7593e5c49cea2957f574a266a29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:09.780279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a0a7593e5c49cea2957f574a266a29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:09.795309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a0a7593e5c49cea2957f574a266a29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:09.802905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a0a7593e5c49cea2957f574a266a29'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:09.809078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1226/2454] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:09 2022] DELETE /identity/v3/users/87a0a7593e5c49cea2957f574a266a29 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:09.813600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.813909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.814196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.834170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:09.835676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:09.896623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1229/2455] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:09.907732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:09.909205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:09.967641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ba730-ddf4-4814-9fab-2b85213eca1c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:09.972085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9ba11-f0ae-45b1-a676-fca4e29ea376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:09.972688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ba11-f0ae-45b1-a676-fca4e29ea376 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:09.972970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ba11-f0ae-45b1-a676-fca4e29ea376 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:09.973241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9ba11-f0ae-45b1-a676-fca4e29ea376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:09.982127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9ba11-f0ae-45b1-a676-fca4e29ea376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:09.983552 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9ba11-f0ae-45b1-a676-fca4e29ea376 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:09.995468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1227/2456] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:12:10.276605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.278156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:10.298897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784290e5-e5f3-4245-9263-934a6a466a25 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.303879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.304550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.304828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.305118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] PATH_INFO: `/v3/projects/57a726146b784308aadcbf92eb524483` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.306244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=57a726146b784308aadcbf92eb524483)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.308176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.348632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:10.361922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57a726146b784308aadcbf92eb524483', 'request_id': 'req-0e7f3905-0c42-48da-9832-2bffeca93f4e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:10.368514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57a726146b784308aadcbf92eb524483', 'request_id': 'req-0e7f3905-0c42-48da-9832-2bffeca93f4e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:10.374056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57a726146b784308aadcbf92eb524483', 'request_id': 'req-0e7f3905-0c42-48da-9832-2bffeca93f4e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:10.386710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] The token cache is being invalidate because project 57a726146b784308aadcbf92eb524483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:10.386995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:10.387605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-0e7f3905-0c42-48da-9832-2bffeca93f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:10.389130 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1230/2457] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:10 2022] DELETE /identity/v3/projects/57a726146b784308aadcbf92eb524483 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:10.394221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.394738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.394834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.416347 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:10.417716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:10.479605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1228/2458] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:12:10.488464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.488718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.488991 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.512650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:10.513886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:10.522604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1231/2459] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:10.526885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.537560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1dac1a-405c-49f7-afe3-3121c7fc595e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.542177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.542772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.543051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.543317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.543961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:10.544291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.545890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.557923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1229/2460] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:12:10.562231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.572994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7618062b-10a6-48f8-a19b-66908b94aead None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.579556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.580329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.580617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.580887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.581548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.583462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.612417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1232/2461] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:10 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:10.616740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.627134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9d03-77a5-4be3-b6f9-2a26eddef18d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.634851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:10.691073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.691671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.691945 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.692242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.692861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.694761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.714809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1230/2462] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:10 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:10.719780 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.730648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d83a1-c051-4fdf-a2e1-7f52d01c7b31 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.736294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.736896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.737173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.737440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.738079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.739670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.749962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1233/2463] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:12:10.754295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.764060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0e32e8-086d-41e2-9c22-33e79867e33c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.768948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a61c74-ead0-43a8-99e3-33d364ee9d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.769534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a61c74-ead0-43a8-99e3-33d364ee9d36 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.769810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a61c74-ead0-43a8-99e3-33d364ee9d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.770081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a61c74-ead0-43a8-99e3-33d364ee9d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.770723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a61c74-ead0-43a8-99e3-33d364ee9d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.772381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a61c74-ead0-43a8-99e3-33d364ee9d36 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.784921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1231/2464] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:10.789279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:10.800259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa4623-d008-42a9-a693-54f9904b904f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:10.805316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:10.805986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.806266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.806545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] PATH_INFO: `/v3/projects/4c7d057ffada475897f2e3cbba817093/users/377a77bc2da9465f974d3ae05fc6a4b8/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.821974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7d057ffada475897f2e3cbba817093, user_id=377a77bc2da9465f974d3ae05fc6a4b8, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:10.824711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:10.838090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1234/2465] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:10 2022] PUT /identity/v3/projects/4c7d057ffada475897f2e3cbba817093/users/377a77bc2da9465f974d3ae05fc6a4b8/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:10.850012 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:10.850294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:10.850564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:10.881277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] MFA Rules not processed for user `377a77bc2da9465f974d3ae05fc6a4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:10.882475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:10.948418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1232/2466] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:10.952989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.015128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e7000-a2e6-4a8a-8b68-b65eaa8b482c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.019818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.020455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.020736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.021102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.021622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.023497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.050766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1235/2467] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:10 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:11.055467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.066501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f378c2c-ff47-47fd-af02-2fa69e548e16 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.125020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.126273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.126569 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.126860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.127584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.129527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.151592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1233/2468] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:11 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:11.155156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.165862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaec59c-f6cf-474e-b8cb-86ccad2d4261 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.170545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.171144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.171425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.171697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.172941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.174528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.184950 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1236/2469] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:11.190218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.201010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070c3d5-e598-45db-af9a-8a892855eeef admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.206367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.207030 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.207313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.207740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] PATH_INFO: `/v3/projects/3b6b3e594b204352a87489b721757037/users/fa936fa45d7b4774864f49cdb78b4c82/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.211133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:11.225532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6b3e594b204352a87489b721757037, user_id=fa936fa45d7b4774864f49cdb78b4c82, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.228235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.241627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1234/2470] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:11 2022] PUT /identity/v3/projects/3b6b3e594b204352a87489b721757037/users/fa936fa45d7b4774864f49cdb78b4c82/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:11.245837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.256359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d68223-5556-4133-adae-8c57423ac4b4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.290905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:11.319242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fade77-e89c-4e17-95a0-db1ef8e5931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.319847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fade77-e89c-4e17-95a0-db1ef8e5931d admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.320167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fade77-e89c-4e17-95a0-db1ef8e5931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.320446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fade77-e89c-4e17-95a0-db1ef8e5931d admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.321092 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fade77-e89c-4e17-95a0-db1ef8e5931d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.322673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fade77-e89c-4e17-95a0-db1ef8e5931d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.332768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1237/2471] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:12:11.337125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.348585 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467b346-3d30-4bb3-9cd7-cd1a958b07ec admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.353199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.353849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.354128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.354401 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] PATH_INFO: `/v3/projects/3b6b3e594b204352a87489b721757037/users/fa936fa45d7b4774864f49cdb78b4c82/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.356353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6b3e594b204352a87489b721757037, user_id=fa936fa45d7b4774864f49cdb78b4c82, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.358965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.369939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1235/2472] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:11 2022] PUT /identity/v3/projects/3b6b3e594b204352a87489b721757037/users/fa936fa45d7b4774864f49cdb78b4c82/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:11.380241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.380529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.380804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.410022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] MFA Rules not processed for user `fa936fa45d7b4774864f49cdb78b4c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:11.411247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:11.481146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1238/2473] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:11 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:11.493609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.495628 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:11.559141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90922c-383e-4fc4-81f9-34a4b2cbc9de admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.563138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.563433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.563772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.564101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.573059 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.574480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.583741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1236/2474] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:11.660328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:11.669967 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68982e8-a7fa-46d3-85d3-955a4adeb6eb None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:11.673761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c626d-1205-494f-a90a-65dac9691c15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:11.674351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c626d-1205-494f-a90a-65dac9691c15 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:11.674634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c626d-1205-494f-a90a-65dac9691c15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:11.674907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c626d-1205-494f-a90a-65dac9691c15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:11.733016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c626d-1205-494f-a90a-65dac9691c15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:11.734504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c626d-1205-494f-a90a-65dac9691c15 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:11.743998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1239/2475] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:14.492637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:14.554782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cb128b-f848-4f46-b68b-2e65d18326cc service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:14.559412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:14.560051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:14.560331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:14.560600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] PATH_INFO: `/v3/users/377a77bc2da9465f974d3ae05fc6a4b8` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:14.561707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=377a77bc2da9465f974d3ae05fc6a4b8)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:14.564564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:14.612283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377a77bc2da9465f974d3ae05fc6a4b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:14.612940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377a77bc2da9465f974d3ae05fc6a4b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:14.620417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377a77bc2da9465f974d3ae05fc6a4b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:14.632795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377a77bc2da9465f974d3ae05fc6a4b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:14.637308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:14.641472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377a77bc2da9465f974d3ae05fc6a4b8'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:14.648508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1237/2476] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:14 2022] DELETE /identity/v3/users/377a77bc2da9465f974d3ae05fc6a4b8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:14.652978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:14.653483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:14.653588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:14.674591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:14.675795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:14.723638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:14.735789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1240/2477] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:14.745519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:14.747068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:14.806987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aaa7cc-b679-4fe4-b935-0b3b97f60ff1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:14.810580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:14.811188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:14.811464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:14.811734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:14.821331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:14.822739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:14.833082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1238/2478] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:15.114951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:15.116723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:15.133073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f67fbc7-e943-412a-ad18-55c24a2878db None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:15.138306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:15.139151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:15.139606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:15.139936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] PATH_INFO: `/v3/projects/4c7d057ffada475897f2e3cbba817093` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:15.141498 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7d057ffada475897f2e3cbba817093)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:15.143386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:15.190455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7d057ffada475897f2e3cbba817093', 'request_id': 'req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.201458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7d057ffada475897f2e3cbba817093', 'request_id': 'req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.208208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7d057ffada475897f2e3cbba817093', 'request_id': 'req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.223645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] The token cache is being invalidate because project 4c7d057ffada475897f2e3cbba817093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:15.223938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.225618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.227299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1241/2479] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:15 2022] DELETE /identity/v3/projects/4c7d057ffada475897f2e3cbba817093 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:15.232070 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:15.242475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117915bc-d9b5-4757-bc94-cbdd13bfa8de service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:15.298729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:15.299657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:15.299936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:15.300238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] PATH_INFO: `/v3/users/fa936fa45d7b4774864f49cdb78b4c82` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:15.301360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa936fa45d7b4774864f49cdb78b4c82)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:15.303281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:15.351475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa936fa45d7b4774864f49cdb78b4c82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.352260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa936fa45d7b4774864f49cdb78b4c82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.361265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa936fa45d7b4774864f49cdb78b4c82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.373275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa936fa45d7b4774864f49cdb78b4c82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.380199 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-eb96d2f9-1733-48af-bc0a-e21e88ba4103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa936fa45d7b4774864f49cdb78b4c82'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.386003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1239/2480] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:15 2022] DELETE /identity/v3/users/fa936fa45d7b4774864f49cdb78b4c82 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:15.662339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:15.663790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:15.677175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5c44f-45ae-4063-9dc8-6f2232bb4a7d admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:15.744692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:15.745337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:15.745616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:15.745893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] PATH_INFO: `/v3/projects/3b6b3e594b204352a87489b721757037` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:15.747038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6b3e594b204352a87489b721757037)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:15.749220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:15.798242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6b3e594b204352a87489b721757037', 'request_id': 'req-219ace21-cd2b-4ada-b064-83df443e427b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.805497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6b3e594b204352a87489b721757037', 'request_id': 'req-219ace21-cd2b-4ada-b064-83df443e427b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.812124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6b3e594b204352a87489b721757037', 'request_id': 'req-219ace21-cd2b-4ada-b064-83df443e427b'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.825234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] The token cache is being invalidate because project 3b6b3e594b204352a87489b721757037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:15.825520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.826142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-219ace21-cd2b-4ada-b064-83df443e427b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:15.827632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1242/2481] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:15 2022] DELETE /identity/v3/projects/3b6b3e594b204352a87489b721757037 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:15.833772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:15.834289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:15.834289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:15.854506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:15.855486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:15.866200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:15.919247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1240/2482] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:15.929383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:15.929383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:15.929689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:15.949034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:15.950279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:15.958680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1243/2483] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:15.962868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:15.973189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66afa67-49eb-47c0-9a1a-fd20fffaf3c7 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:15.977635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:15.978223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:15.978498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:15.978766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:15.979430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:15.979730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:15.981380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:15.993895 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1241/2484] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:12:15.998165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.002596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:16.008735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbb22c-adf1-411a-adb2-d6aeb89e2463 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.014710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.015307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.015589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.015956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.016696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.018663 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.045807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1244/2485] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:15 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:16.050808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.060766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec368-3a23-4393-aee0-76e9d7ef2a81 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.118621 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.119146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.119424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.119699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.120407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.122279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.144266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1242/2486] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:16 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:16.148518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.159921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ffe01-8455-422e-8f8f-4f9e35d5f6eb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.164617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.165212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.165495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.165767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.166407 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.167990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.178167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1245/2487] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:16.182132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.192078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05354b30-f657-4f62-8d74-d55dc38bf55e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.196763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34a5e8-5f6b-43d6-8958-b430bda0d741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.197351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34a5e8-5f6b-43d6-8958-b430bda0d741 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.197633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34a5e8-5f6b-43d6-8958-b430bda0d741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.197915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34a5e8-5f6b-43d6-8958-b430bda0d741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.198556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34a5e8-5f6b-43d6-8958-b430bda0d741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.200240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34a5e8-5f6b-43d6-8958-b430bda0d741 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.211277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1243/2488] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:16.215481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.227112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852166ed-0222-4c84-bd29-8a626cacdfea admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.231945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.232625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.232903 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.233173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] PATH_INFO: `/v3/projects/23bcab9ac7bc48a3a9c50f14c4a1a09c/users/450c4b840a274115940addc4bcca7fb1/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.248539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23bcab9ac7bc48a3a9c50f14c4a1a09c, user_id=450c4b840a274115940addc4bcca7fb1, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.251204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.263774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1246/2489] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:16 2022] PUT /identity/v3/projects/23bcab9ac7bc48a3a9c50f14c4a1a09c/users/450c4b840a274115940addc4bcca7fb1/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:16.274649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.275083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.275363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.304518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] MFA Rules not processed for user `450c4b840a274115940addc4bcca7fb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:16.305743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:16.372058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1244/2490] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:16.376576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.440427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ac4f3-5ce9-4876-89a2-dc0b6c78b7b7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.445086 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.445682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.445958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.446231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.446873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.448779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.477871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1247/2491] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:16 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:16.481770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.493743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd6e5e9-185e-436e-baf9-c4c920769402 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.502787 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:16.557573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.558176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.558458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.558730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.559371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.561387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.586635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1245/2492] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:16 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:16.587886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.599265 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669315ea-c0d0-4824-8ed3-d581425ca34f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.604078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.604686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.604965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.605240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.605912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.607525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.619239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1248/2493] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:16.623440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.633512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b30406-38c9-4d34-99e5-e90add3e13c9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.638208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.638854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.639133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.639406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] PATH_INFO: `/v3/projects/184ed5ca606b4c97980fd15ac92e8af2/users/87807b43442b4cabab4c81c71bcbee7b/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.655302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] RBAC: Authorizing `identity:create_grant(project_id=184ed5ca606b4c97980fd15ac92e8af2, user_id=87807b43442b4cabab4c81c71bcbee7b, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.658647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.673545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1246/2494] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:16 2022] PUT /identity/v3/projects/184ed5ca606b4c97980fd15ac92e8af2/users/87807b43442b4cabab4c81c71bcbee7b/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:16.677618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.688371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6da39f-110b-4ac1-a90d-b0df60675ee9 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.749269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183338cf-04bf-49f6-8458-125ec5a1bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.749901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183338cf-04bf-49f6-8458-125ec5a1bec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.750865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183338cf-04bf-49f6-8458-125ec5a1bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.750983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183338cf-04bf-49f6-8458-125ec5a1bec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.751604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183338cf-04bf-49f6-8458-125ec5a1bec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.753217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183338cf-04bf-49f6-8458-125ec5a1bec0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.763118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1249/2495] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:16.768457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.779657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcc633-625e-46ca-aa70-acc1ead7abea admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:16.785503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:16.786351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.786645 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.786975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] PATH_INFO: `/v3/projects/184ed5ca606b4c97980fd15ac92e8af2/users/87807b43442b4cabab4c81c71bcbee7b/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.789192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=184ed5ca606b4c97980fd15ac92e8af2, user_id=87807b43442b4cabab4c81c71bcbee7b, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:16.791948 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:16.803106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1247/2496] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:16 2022] PUT /identity/v3/projects/184ed5ca606b4c97980fd15ac92e8af2/users/87807b43442b4cabab4c81c71bcbee7b/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:16.814364 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:16.814651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:16.814929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:16.845182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] MFA Rules not processed for user `87807b43442b4cabab4c81c71bcbee7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:16.846388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:16.853081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:16.939410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1250/2497] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:12:16.952078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:16.953852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:17.013255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab1404-8544-4bd7-829c-00830be8c9a4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:17.017292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:17.017897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:17.018172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:17.018519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:17.074899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:17.076362 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:17.085131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1248/2498] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:22.735589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:22.737071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:22.748835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc5f2d-64f8-46b7-8633-3599affcdfb5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:22.752431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e6270-1e11-43c0-b3b5-16f395e9aa79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:22.753031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6270-1e11-43c0-b3b5-16f395e9aa79 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:22.753313 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6270-1e11-43c0-b3b5-16f395e9aa79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:22.753587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e6270-1e11-43c0-b3b5-16f395e9aa79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:22.762932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6270-1e11-43c0-b3b5-16f395e9aa79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:22.764404 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e6270-1e11-43c0-b3b5-16f395e9aa79 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:22.777542 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1251/2499] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:23.531953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:23.533733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:23.554732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:23.601879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77f2ca-b044-4db8-9471-8d343c771bb2 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:23.606416 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:23.607040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:23.607320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:23.607593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] PATH_INFO: `/v3/users/b14c5d9d54064be88abdbd0b9d8b24dd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:23.608730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14c5d9d54064be88abdbd0b9d8b24dd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:23.611211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:23.658388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14c5d9d54064be88abdbd0b9d8b24dd'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:23.659105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14c5d9d54064be88abdbd0b9d8b24dd'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:23.666875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14c5d9d54064be88abdbd0b9d8b24dd'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:23.679424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14c5d9d54064be88abdbd0b9d8b24dd'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:23.686285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14c5d9d54064be88abdbd0b9d8b24dd'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:23.692123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1249/2500] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:23 2022] DELETE /identity/v3/users/b14c5d9d54064be88abdbd0b9d8b24dd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:23.696909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:23.697390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:23.697871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:23.719548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:23.720804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:23.779758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1252/2501] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:23.791277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:23.792447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:23.856056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41892695-8764-4a05-a35c-2a0a235866b7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:23.859871 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42c1e3-982e-4a9d-b335-18819ebfa035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:23.860500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42c1e3-982e-4a9d-b335-18819ebfa035 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:23.860776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42c1e3-982e-4a9d-b335-18819ebfa035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:23.861047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42c1e3-982e-4a9d-b335-18819ebfa035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:23.871380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42c1e3-982e-4a9d-b335-18819ebfa035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:23.872849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae42c1e3-982e-4a9d-b335-18819ebfa035 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:23.882594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1250/2502] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:24.155833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.157629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:24.172270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f789e-1376-4591-b2d7-1690b159bd8d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.177154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.177917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.178220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.178627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] PATH_INFO: `/v3/projects/8837c791843f49339e74e9bc5035f617` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.180391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8837c791843f49339e74e9bc5035f617)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.182345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.233271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8837c791843f49339e74e9bc5035f617', 'request_id': 'req-e7029dcd-213f-4d46-b28c-f80576e3f378'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:24.239458 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8837c791843f49339e74e9bc5035f617', 'request_id': 'req-e7029dcd-213f-4d46-b28c-f80576e3f378'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:24.245025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8837c791843f49339e74e9bc5035f617', 'request_id': 'req-e7029dcd-213f-4d46-b28c-f80576e3f378'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:24.258381 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] The token cache is being invalidate because project 8837c791843f49339e74e9bc5035f617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:24.258672 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:24.259295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e7029dcd-213f-4d46-b28c-f80576e3f378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:24.260843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1253/2503] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:24 2022] DELETE /identity/v3/projects/8837c791843f49339e74e9bc5035f617 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:24.266313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.266768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.267052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.287897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:24.289190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:24.323299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:24.350131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1251/2504] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:24.357947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.358230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.358507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.379037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:24.380304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:24.388245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1254/2505] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:24.392532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.402940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd586fb-0bb2-4dce-9df0-2765d7174a8c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.408433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.409037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.409318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.409618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.410284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:24.410589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.412208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.424500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1252/2506] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:12:24.429018 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.434680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:24.439514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff124af-56ca-48f5-b325-02bc4faa2bed None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.445517 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.446283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.446746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.447064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.447922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.449856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.477349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1255/2507] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:24 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:24.481387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.491942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79146762-f9b2-4b3f-834c-1a44eecac4fe admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.552811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.553498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.553611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.553850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.554491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.556579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.581231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1253/2508] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:24 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:24.588947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.600561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fabc1e-0596-4736-811d-bae47dd12d24 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.605484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.606244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.606551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.606802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.607445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.609628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.619964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1256/2509] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:12:24.624316 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.635331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139d336-afb3-4954-b5dc-2b318d2cf10f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.640136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c06fb-180b-443b-ac60-e1db2e4b60d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.640798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c06fb-180b-443b-ac60-e1db2e4b60d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.641076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c06fb-180b-443b-ac60-e1db2e4b60d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.641353 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c06fb-180b-443b-ac60-e1db2e4b60d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.642003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c06fb-180b-443b-ac60-e1db2e4b60d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.644019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c06fb-180b-443b-ac60-e1db2e4b60d8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.655090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1254/2510] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:24.659520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.669655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25088a-1d7a-4365-9977-b6e29b3634f2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.674880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.675525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.675822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.676123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] PATH_INFO: `/v3/projects/47f7d8433ff14004a40ca90ac052199f/users/37b8d78a8658482da897e742d8136bf0/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.691858 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f7d8433ff14004a40ca90ac052199f, user_id=37b8d78a8658482da897e742d8136bf0, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.694808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.708772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1257/2511] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:24 2022] PUT /identity/v3/projects/47f7d8433ff14004a40ca90ac052199f/users/37b8d78a8658482da897e742d8136bf0/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:24.776553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.776769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.777025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.804987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] MFA Rules not processed for user `37b8d78a8658482da897e742d8136bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:24.806190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:24.878915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1255/2512] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:24.887294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:24.889114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:24.946866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524236d3-51e8-4c15-be2b-2de06e6d44cb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:24.950494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c11ef-4fd4-4b36-a983-83dc2f0c456c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:24.951090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c11ef-4fd4-4b36-a983-83dc2f0c456c service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:24.951365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c11ef-4fd4-4b36-a983-83dc2f0c456c service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:24.951673 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c11ef-4fd4-4b36-a983-83dc2f0c456c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:24.960583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c11ef-4fd4-4b36-a983-83dc2f0c456c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:24.961996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c11ef-4fd4-4b36-a983-83dc2f0c456c service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:24.970845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1258/2513] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:34.871704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:34.966653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:40.505657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:40.567002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f12b1-dd91-4041-99fc-17cbd9321534 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:40.571659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:40.572307 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:40.572588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:40.572858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] PATH_INFO: `/v3/users/37b8d78a8658482da897e742d8136bf0` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:40.573968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=37b8d78a8658482da897e742d8136bf0)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:40.576136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:40.621461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37b8d78a8658482da897e742d8136bf0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:40.622148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37b8d78a8658482da897e742d8136bf0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:40.630369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37b8d78a8658482da897e742d8136bf0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:40.643670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37b8d78a8658482da897e742d8136bf0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:40.650394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37b8d78a8658482da897e742d8136bf0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:40.656655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1256/2514] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:40 2022] DELETE /identity/v3/users/37b8d78a8658482da897e742d8136bf0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:40.661216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:40.661743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:40.661841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:40.680969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:40.682173 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:40.741628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1259/2515] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:40.752244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:40.753640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:40.814781 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3334c25b-dc95-4758-aefb-31224688e4ef admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:40.819023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:40.819637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:40.819922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:40.820233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:40.831252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:40.832776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:40.842354 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1257/2516] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:41.112260 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:41.113844 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:41.126748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaa677-0e42-4119-9826-a60f634774c8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:41.131944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:41.132627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:41.133093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:41.133368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] PATH_INFO: `/v3/projects/47f7d8433ff14004a40ca90ac052199f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:41.134500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f7d8433ff14004a40ca90ac052199f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:41.136562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:41.187989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f7d8433ff14004a40ca90ac052199f', 'request_id': 'req-1c814791-6a18-46b0-b248-e40eec165d86'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:41.194176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f7d8433ff14004a40ca90ac052199f', 'request_id': 'req-1c814791-6a18-46b0-b248-e40eec165d86'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:41.199660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f7d8433ff14004a40ca90ac052199f', 'request_id': 'req-1c814791-6a18-46b0-b248-e40eec165d86'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:41.211727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] The token cache is being invalidate because project 47f7d8433ff14004a40ca90ac052199f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:41.212037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:41.212664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1c814791-6a18-46b0-b248-e40eec165d86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:41.214148 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1260/2517] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:41 2022] DELETE /identity/v3/projects/47f7d8433ff14004a40ca90ac052199f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:47.707732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:47.709221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:47.722625 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66c854-ac80-4eef-acf6-ec997b5dd0ff service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:47.740392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:47.784125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:47.785225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:47.785690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:47.786179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] PATH_INFO: `/v3/users/450c4b840a274115940addc4bcca7fb1` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:47.788644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=450c4b840a274115940addc4bcca7fb1)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:47.792096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:47.838568 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '450c4b840a274115940addc4bcca7fb1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:47.839219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '450c4b840a274115940addc4bcca7fb1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:47.847422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '450c4b840a274115940addc4bcca7fb1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:47.859536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '450c4b840a274115940addc4bcca7fb1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:47.866252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '450c4b840a274115940addc4bcca7fb1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:47.871980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1258/2518] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:47 2022] DELETE /identity/v3/users/450c4b840a274115940addc4bcca7fb1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:47.876696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:47.877232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:47.877232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:47.896604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:47.897791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:47.958454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1261/2519] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:47.968626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:47.970037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:48.026829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82169665-33e7-49ba-8c19-6d7750ff2902 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:48.031096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:48.031688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:48.031977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:48.032292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:48.041159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:48.042573 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:48.052723 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1259/2520] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:48.323982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:48.325503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:48.343851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8572c-7c8e-4574-9647-42382933cf0a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:48.348904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:48.349543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:48.349824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:48.350098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] PATH_INFO: `/v3/projects/23bcab9ac7bc48a3a9c50f14c4a1a09c` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:48.351312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=23bcab9ac7bc48a3a9c50f14c4a1a09c)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:48.353176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:48.374805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:48.402108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23bcab9ac7bc48a3a9c50f14c4a1a09c', 'request_id': 'req-3368d186-d3df-4f60-b993-58a98f8ff13f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.410402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23bcab9ac7bc48a3a9c50f14c4a1a09c', 'request_id': 'req-3368d186-d3df-4f60-b993-58a98f8ff13f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.416409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23bcab9ac7bc48a3a9c50f14c4a1a09c', 'request_id': 'req-3368d186-d3df-4f60-b993-58a98f8ff13f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.429053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] The token cache is being invalidate because project 23bcab9ac7bc48a3a9c50f14c4a1a09c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:48.429354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.429963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.431688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1262/2521] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:48 2022] DELETE /identity/v3/projects/23bcab9ac7bc48a3a9c50f14c4a1a09c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:48.435729 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:48.445396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791b790-f829-493c-8891-715e36666a88 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:48.487280 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:48.503086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:48.503665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:48.503935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:48.504235 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] PATH_INFO: `/v3/users/87807b43442b4cabab4c81c71bcbee7b` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:48.505346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=87807b43442b4cabab4c81c71bcbee7b)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:48.507306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:48.552517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87807b43442b4cabab4c81c71bcbee7b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.553185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87807b43442b4cabab4c81c71bcbee7b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.559852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87807b43442b4cabab4c81c71bcbee7b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.572515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87807b43442b4cabab4c81c71bcbee7b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.579768 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81c5fafb-e7c4-4985-9a2e-b80dfc9a4b3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87807b43442b4cabab4c81c71bcbee7b'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.585622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1260/2522] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:48 2022] DELETE /identity/v3/users/87807b43442b4cabab4c81c71bcbee7b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:48.862125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:48.863846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:48.879321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368d186-d3df-4f60-b993-58a98f8ff13f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:48.941621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:48.942242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:48.942539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:48.942793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] PATH_INFO: `/v3/projects/184ed5ca606b4c97980fd15ac92e8af2` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:48.943935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=184ed5ca606b4c97980fd15ac92e8af2)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:48.945853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:48.991585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '184ed5ca606b4c97980fd15ac92e8af2', 'request_id': 'req-69c7ec41-ea2e-41cd-9eec-08aea81083e6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:48.997694 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '184ed5ca606b4c97980fd15ac92e8af2', 'request_id': 'req-69c7ec41-ea2e-41cd-9eec-08aea81083e6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:49.003820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '184ed5ca606b4c97980fd15ac92e8af2', 'request_id': 'req-69c7ec41-ea2e-41cd-9eec-08aea81083e6'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:49.016796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] The token cache is being invalidate because project 184ed5ca606b4c97980fd15ac92e8af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:49.017081 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:49.017698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-69c7ec41-ea2e-41cd-9eec-08aea81083e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:49.019175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1263/2523] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:48 2022] DELETE /identity/v3/projects/184ed5ca606b4c97980fd15ac92e8af2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:49.024247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.024726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.024841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.044982 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:49.046134 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:49.104564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1261/2524] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:49.113441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.113731 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.114002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.134426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:49.135968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:49.143710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1264/2525] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:12:49.148088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.158424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c960c-51a9-415d-9353-0a2820f254e9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.163084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.163680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.163960 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.164272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.164921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:49.165364 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.166950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.179342 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1262/2526] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:12:49.184311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.195500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882448-2b2b-4585-a9b9-39c98da6df62 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.200612 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.201270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.202485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.202485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.202902 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.205172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.235075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1265/2527] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:49.239358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.250105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c7d89-8905-472d-a433-15dfcdc154cf admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.296893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:49.308116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.308742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.309019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.309291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.309927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.312642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.334437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1263/2528] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:49.338536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.349373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60f5953-88f7-49fa-a260-b7d83088266a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.353798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.354388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.354664 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.354933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.355570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.357183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.360262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:49.372413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1266/2529] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:49.375293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.386634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb66dc2-11ee-41a4-9746-67821d098c80 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.391352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2cd80-a3d6-4929-9517-eaa78906a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.391949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2cd80-a3d6-4929-9517-eaa78906a255 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.392259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2cd80-a3d6-4929-9517-eaa78906a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.392534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2cd80-a3d6-4929-9517-eaa78906a255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.393172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2cd80-a3d6-4929-9517-eaa78906a255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.394764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2cd80-a3d6-4929-9517-eaa78906a255 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.406484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1264/2530] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:49.410707 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.421229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d6f46-9bdf-41e0-8fa8-f40029c2f9cf admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.425850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.426491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.426771 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.427049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] PATH_INFO: `/v3/projects/1ecf59db12c940418c4bc726da9dd539/users/daf2ae12322a46ad858e18c823208c79/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.443059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecf59db12c940418c4bc726da9dd539, user_id=daf2ae12322a46ad858e18c823208c79, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.445713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.459001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1267/2531] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:49 2022] PUT /identity/v3/projects/1ecf59db12c940418c4bc726da9dd539/users/daf2ae12322a46ad858e18c823208c79/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:12:49.470313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738137-7653-47fe-ac77-fe3859228882 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.470597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738137-7653-47fe-ac77-fe3859228882 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.470865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85738137-7653-47fe-ac77-fe3859228882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.500162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-85738137-7653-47fe-ac77-fe3859228882 None None] MFA Rules not processed for user `daf2ae12322a46ad858e18c823208c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:49.501394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-85738137-7653-47fe-ac77-fe3859228882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:49.565049 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1265/2532] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:12:49.570170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.633742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1379c9-cef2-4e0d-b6e1-69fe7c32feac admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.638607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.639218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.639503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.639807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.640525 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.642432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.670161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1268/2533] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:49.674146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85738137-7653-47fe-ac77-fe3859228882 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.684928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85738137-7653-47fe-ac77-fe3859228882 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.740886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.741491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.741773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.742048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.742694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.744649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.765910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1266/2534] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:49.769243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.780176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1347994-a13d-4ad8-9675-ce75fe57067a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.784749 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.785350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.785628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.785905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.786545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.788166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.799733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1269/2535] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:49.803978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.814035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7549c17-436e-4ef0-b93b-c3eac869d11d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.818893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.819508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.819784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.820091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] PATH_INFO: `/v3/projects/f3e4b22f8bee4485a9e3a32a9d6a6717/users/c4781fb72c8e48c5af6e8feadea43254/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.835227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e4b22f8bee4485a9e3a32a9d6a6717, user_id=c4781fb72c8e48c5af6e8feadea43254, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.837963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.851206 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1267/2536] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:49 2022] PUT /identity/v3/projects/f3e4b22f8bee4485a9e3a32a9d6a6717/users/c4781fb72c8e48c5af6e8feadea43254/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:49.855131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.865095 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cde25-b244-4194-a37a-c90d6020eae0 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.923305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5c3304-7845-4ad9-b0d3-54fe3e73331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.924041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c3304-7845-4ad9-b0d3-54fe3e73331a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.924206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c3304-7845-4ad9-b0d3-54fe3e73331a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.924478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5c3304-7845-4ad9-b0d3-54fe3e73331a admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.925120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c3304-7845-4ad9-b0d3-54fe3e73331a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.926691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5c3304-7845-4ad9-b0d3-54fe3e73331a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.937014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1270/2537] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:49.941245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:49.952048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6919b6ce-2ec2-4173-ac82-669d4a34e636 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:49.956789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:49.957557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.957844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.958117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] PATH_INFO: `/v3/projects/f3e4b22f8bee4485a9e3a32a9d6a6717/users/c4781fb72c8e48c5af6e8feadea43254/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:49.960050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3e4b22f8bee4485a9e3a32a9d6a6717, user_id=c4781fb72c8e48c5af6e8feadea43254, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:49.962791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:49.974183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1268/2538] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:49 2022] PUT /identity/v3/projects/f3e4b22f8bee4485a9e3a32a9d6a6717/users/c4781fb72c8e48c5af6e8feadea43254/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:12:49.986915 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:49.987281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:49.987629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:50.017238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] MFA Rules not processed for user `c4781fb72c8e48c5af6e8feadea43254`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:50.018486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:50.052974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:50.099711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1271/2539] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:50.111384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:50.113906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:50.128855 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:50.179753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c99529-ac85-4025-93b5-af74781df0c8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:50.184080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec4d232-be3c-4cbb-82d6-e7b565f7b785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:50.185139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d232-be3c-4cbb-82d6-e7b565f7b785 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:50.185426 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d232-be3c-4cbb-82d6-e7b565f7b785 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:50.185968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec4d232-be3c-4cbb-82d6-e7b565f7b785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:50.195800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d232-be3c-4cbb-82d6-e7b565f7b785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:50.197276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec4d232-be3c-4cbb-82d6-e7b565f7b785 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:50.209654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1269/2540] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:50.246860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:50.314923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0093b14-13e4-4c91-9112-cc016cce89b0 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:50.320117 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:50.320753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:50.321034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:50.321307 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] PATH_INFO: `/v3/users/daf2ae12322a46ad858e18c823208c79` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:50.322464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf2ae12322a46ad858e18c823208c79)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:50.324437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:50.376610 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf2ae12322a46ad858e18c823208c79'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.385296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf2ae12322a46ad858e18c823208c79'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.395293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf2ae12322a46ad858e18c823208c79'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.415582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf2ae12322a46ad858e18c823208c79'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.416741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf2ae12322a46ad858e18c823208c79'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.427762 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1272/2541] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:50 2022] DELETE /identity/v3/users/daf2ae12322a46ad858e18c823208c79 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:50.433972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:50.434383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:50.434766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:50.454478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:50.455654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:50.513227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1270/2542] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:50.523796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:50.525280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:50.578732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:50.582836 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d41c193-fa8c-4d25-bb7d-b17e8546fcde admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:50.587772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:50.588573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:50.589088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:50.589637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:50.599649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:50.601137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:50.611704 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1273/2543] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:50.878290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:50.879828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:50.892438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06919ae3-537e-4e16-b640-1e1d0ae8e577 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:50.897036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:50.897665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:50.897941 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:50.898210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] PATH_INFO: `/v3/projects/1ecf59db12c940418c4bc726da9dd539` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:50.899383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecf59db12c940418c4bc726da9dd539)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:50.901332 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:50.949604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecf59db12c940418c4bc726da9dd539', 'request_id': 'req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.956967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecf59db12c940418c4bc726da9dd539', 'request_id': 'req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.962712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecf59db12c940418c4bc726da9dd539', 'request_id': 'req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.974689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] The token cache is being invalidate because project 1ecf59db12c940418c4bc726da9dd539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:50.974976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.975622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:50.977149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1271/2544] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:50 2022] DELETE /identity/v3/projects/1ecf59db12c940418c4bc726da9dd539 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:50.981092 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:50.990976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d0c7-a02f-428a-acab-4c30f6866029 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.047787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.048443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.048727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.049000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] PATH_INFO: `/v3/users/c4781fb72c8e48c5af6e8feadea43254` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.050174 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4781fb72c8e48c5af6e8feadea43254)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:51.052185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:51.097711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4781fb72c8e48c5af6e8feadea43254'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.104561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4781fb72c8e48c5af6e8feadea43254'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.112497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4781fb72c8e48c5af6e8feadea43254'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.125565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4781fb72c8e48c5af6e8feadea43254'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.126227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5c478062-4185-47c9-8320-41f764fda91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4781fb72c8e48c5af6e8feadea43254'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.132104 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1274/2545] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:50 2022] DELETE /identity/v3/users/c4781fb72c8e48c5af6e8feadea43254 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:51.422627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.424118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:51.440453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792ec2b-2e5c-4e25-8b88-16285ee1d2c3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.479692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:51.505143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.505815 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.506105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.506384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] PATH_INFO: `/v3/projects/f3e4b22f8bee4485a9e3a32a9d6a6717` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.507532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3e4b22f8bee4485a9e3a32a9d6a6717)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:51.509486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:51.558035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3e4b22f8bee4485a9e3a32a9d6a6717', 'request_id': 'req-002159e3-9b88-4521-a059-6bf0c36ffaf4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.566474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3e4b22f8bee4485a9e3a32a9d6a6717', 'request_id': 'req-002159e3-9b88-4521-a059-6bf0c36ffaf4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.574065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3e4b22f8bee4485a9e3a32a9d6a6717', 'request_id': 'req-002159e3-9b88-4521-a059-6bf0c36ffaf4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.587978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] The token cache is being invalidate because project f3e4b22f8bee4485a9e3a32a9d6a6717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:51.588405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.589161 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-002159e3-9b88-4521-a059-6bf0c36ffaf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:51.591230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1272/2546] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:51 2022] DELETE /identity/v3/projects/f3e4b22f8bee4485a9e3a32a9d6a6717 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:51.595965 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.596387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.596783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.629088 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:51.630287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:51.688855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1275/2547] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:51.696685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.696968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.697242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.717720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:51.718914 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:51.727168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1273/2548] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:51.731683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.742089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3a029-9d49-4ae4-a3d9-6ba78dacd5d7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.746756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.747366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.747647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.747921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.749269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:51.749601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:51.751238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:51.758782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:51.765620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1276/2549] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:12:51.769509 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.778981 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f7ea9-9211-4a14-b63a-430523028488 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.784321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.785022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.785302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.785581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.786323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:51.788330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:51.818760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1274/2550] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:51 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:51.827295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.829609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.832157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:51.839284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b0988-860a-468f-bc96-aec1f6c23e74 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.902593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.903631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.903631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.903817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.904424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:51.906818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:51.916152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b93ca1-044b-4c81-bd4c-b95d8f56dcca admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.923727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.924381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.924878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.925164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.928440 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:51.931873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1277/2551] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:51 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:51.936459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.952121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a0611-9651-4552-8f1b-21341c2208dc admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.957170 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.957857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.958203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.958545 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.959287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:51.961005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:51.976778 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1278/2552] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:12:51.978590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:51.979972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:51.992442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351e6a8-7102-4194-8444-c1edb2233741 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:51.997353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:51.997951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:51.998786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:51.999488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:51.999910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.001529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.009399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.011068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e68d60-7cea-4ea4-a15b-ae0752561c3d service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.015847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1279/2553] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:52.016978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.022349 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1275/2554] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:52.027691 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb010c-9b58-407f-834f-b61ef678ad21 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.033247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.033860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.034165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.034440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] PATH_INFO: `/v3/projects/9a3133c2dd3d4c81ae19263ba3eeb452/users/9cddfbbc9c8e443180c74d3caf7e1b73/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.051056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a3133c2dd3d4c81ae19263ba3eeb452, user_id=9cddfbbc9c8e443180c74d3caf7e1b73, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.053784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.070914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1280/2555] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:52 2022] PUT /identity/v3/projects/9a3133c2dd3d4c81ae19263ba3eeb452/users/9cddfbbc9c8e443180c74d3caf7e1b73/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:12:52.084852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.085310 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.086326 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.118862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] MFA Rules not processed for user `9cddfbbc9c8e443180c74d3caf7e1b73`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:52.120234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:52.198004 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1276/2556] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:52.203720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.287311 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5223a0-9c09-41d4-af27-30dcadac978f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.292617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.293229 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.293497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.293820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.294493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.296994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.335007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1281/2557] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:52 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:52.339021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.350939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d3c3b-aeef-429f-869c-618e9c433772 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.447853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.449117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.449406 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.449682 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.450323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.452434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.479761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1277/2558] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:52 2022] POST /identity/v3/users => generated 297 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:52.480990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.492464 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3cc78-527c-4319-a90c-b6715383db87 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.497297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.497852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.498162 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.498438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.499167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.500800 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.511806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1282/2559] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:52.516130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.526382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006e0245-6ab9-4303-9184-c8a732d491c9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.532055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.532843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.533122 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.533393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] PATH_INFO: `/v3/projects/9fb86c06e20547738cf0ad2361eacc3f/users/d645e7449efa4671a960cf7f53a8bdd9/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.549370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb86c06e20547738cf0ad2361eacc3f, user_id=d645e7449efa4671a960cf7f53a8bdd9, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.552073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.565473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1278/2560] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:52 2022] PUT /identity/v3/projects/9fb86c06e20547738cf0ad2361eacc3f/users/d645e7449efa4671a960cf7f53a8bdd9/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:52.569589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.580360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893738aa-5bbd-4e26-a9e5-95cbc11191b9 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.592653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:52.680031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d193f3-87f0-4909-8955-d0d900276d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.680819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d193f3-87f0-4909-8955-d0d900276d01 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.681197 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d193f3-87f0-4909-8955-d0d900276d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.681589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d193f3-87f0-4909-8955-d0d900276d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.682508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d193f3-87f0-4909-8955-d0d900276d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.683994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d193f3-87f0-4909-8955-d0d900276d01 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.696106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1283/2561] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:52.700292 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.711857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa39219-d357-41d5-aebb-e9aeebb93b26 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:52.724931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:52.725737 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.726202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.726596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] PATH_INFO: `/v3/projects/9fb86c06e20547738cf0ad2361eacc3f/users/d645e7449efa4671a960cf7f53a8bdd9/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.728783 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fb86c06e20547738cf0ad2361eacc3f, user_id=d645e7449efa4671a960cf7f53a8bdd9, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:52.731539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:52.746861 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1279/2562] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:52 2022] PUT /identity/v3/projects/9fb86c06e20547738cf0ad2361eacc3f/users/d645e7449efa4671a960cf7f53a8bdd9/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:12:52.760215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:52.760417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:52.760680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:52.817670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] MFA Rules not processed for user `d645e7449efa4671a960cf7f53a8bdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:52.819516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:52.910185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1284/2563] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:52.924090 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:52.926198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:52.950475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:52.998848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61887669-c278-4df8-a252-66218c3cf910 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:53.003337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f80c88-f5d3-4f68-b07c-3cf757a32738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:53.004182 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f80c88-f5d3-4f68-b07c-3cf757a32738 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:53.004626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f80c88-f5d3-4f68-b07c-3cf757a32738 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:53.005058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f80c88-f5d3-4f68-b07c-3cf757a32738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:53.016208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f80c88-f5d3-4f68-b07c-3cf757a32738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:53.017793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f80c88-f5d3-4f68-b07c-3cf757a32738 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:53.028772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1280/2564] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:53.504851 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:53.601578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bdecfd-4ba3-40ea-9d5a-5b50eb1d01fb None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:53.607647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:53.608483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:53.608767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:53.609042 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] PATH_INFO: `/v3/users/9cddfbbc9c8e443180c74d3caf7e1b73` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:53.611078 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cddfbbc9c8e443180c74d3caf7e1b73)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:53.613187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:53.662402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:53.670580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cddfbbc9c8e443180c74d3caf7e1b73'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:53.678079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cddfbbc9c8e443180c74d3caf7e1b73'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:53.686428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cddfbbc9c8e443180c74d3caf7e1b73'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:53.705332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cddfbbc9c8e443180c74d3caf7e1b73'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:53.706147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cddfbbc9c8e443180c74d3caf7e1b73'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:53.713358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1285/2565] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:53 2022] DELETE /identity/v3/users/9cddfbbc9c8e443180c74d3caf7e1b73 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:53.718539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:53.719052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:53.719165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:53.740116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:53.741330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:53.836138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1281/2566] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:53.844126 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:53.846072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:53.940134 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08a75d-0300-43db-918b-2af24abeb2e4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:53.947740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:53.948618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:53.949178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:53.949712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:53.960864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:53.964152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:53.976637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1286/2567] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:54.509077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:54.510496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:54.523578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3755d-4f8a-4791-9a84-a5979690e46f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:54.528505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:54.529130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:54.530179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:54.530543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] PATH_INFO: `/v3/projects/9a3133c2dd3d4c81ae19263ba3eeb452` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:54.531728 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a3133c2dd3d4c81ae19263ba3eeb452)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:54.533718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:54.598859 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a3133c2dd3d4c81ae19263ba3eeb452', 'request_id': 'req-ef2973b4-2010-43d5-812d-01438fff9274'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.607365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a3133c2dd3d4c81ae19263ba3eeb452', 'request_id': 'req-ef2973b4-2010-43d5-812d-01438fff9274'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.613061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a3133c2dd3d4c81ae19263ba3eeb452', 'request_id': 'req-ef2973b4-2010-43d5-812d-01438fff9274'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.625054 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] The token cache is being invalidate because project 9a3133c2dd3d4c81ae19263ba3eeb452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:54.625587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.626447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.628273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1282/2568] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:54 2022] DELETE /identity/v3/projects/9a3133c2dd3d4c81ae19263ba3eeb452 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:54.632274 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:54.643414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c13edd-3132-4cd9-9933-5fd601b18749 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:54.711421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:54.712112 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:54.712393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:54.712667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] PATH_INFO: `/v3/users/d645e7449efa4671a960cf7f53a8bdd9` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:54.713798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d645e7449efa4671a960cf7f53a8bdd9)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:54.715963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:54.769334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd645e7449efa4671a960cf7f53a8bdd9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.784587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd645e7449efa4671a960cf7f53a8bdd9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.789400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:54.794622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd645e7449efa4671a960cf7f53a8bdd9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.809164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd645e7449efa4671a960cf7f53a8bdd9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.809847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-7da4d0fc-15d2-41f8-b55d-6a0df0e12717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd645e7449efa4671a960cf7f53a8bdd9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:54.817183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1287/2569] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:54 2022] DELETE /identity/v3/users/d645e7449efa4671a960cf7f53a8bdd9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:12:55.126095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.127604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:55.143380 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2973b4-2010-43d5-812d-01438fff9274 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.188797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:55.205172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.205819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.206101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.206375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] PATH_INFO: `/v3/projects/9fb86c06e20547738cf0ad2361eacc3f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.207508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fb86c06e20547738cf0ad2361eacc3f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.209496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.257545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fb86c06e20547738cf0ad2361eacc3f', 'request_id': 'req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:55.265455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fb86c06e20547738cf0ad2361eacc3f', 'request_id': 'req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:55.271120 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fb86c06e20547738cf0ad2361eacc3f', 'request_id': 'req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:55.282718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] The token cache is being invalidate because project 9fb86c06e20547738cf0ad2361eacc3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:55.283032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:55.283702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-72559d0a-75cc-4b17-a1e4-391d4d3cf6c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:55.285208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1283/2570] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:55 2022] DELETE /identity/v3/projects/9fb86c06e20547738cf0ad2361eacc3f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:55.290246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.290537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.290816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.311146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:55.312409 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:55.372276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1288/2571] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:55.379953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.380290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.380572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.401278 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:55.402479 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:55.411793 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1284/2572] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:55.416440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.426320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6bb52-d6f5-4617-bc06-6f53d85df80e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.430865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.431457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.431736 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.432024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.432681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:55.432984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.434562 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.446076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1289/2573] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:12:55.451747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.466114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8fdb5-a10e-4bfd-827f-e4b5de4fe507 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.471423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.472048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.472334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.472611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.473246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.475192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.510470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1285/2574] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:55 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:55.515373 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.526099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496f4a3-dafc-4fc1-b8fd-9d840545d753 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.594069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.594986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.595451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.595933 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.596628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.599264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.621963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1290/2575] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:55 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:55.626305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.636709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e5b75-0c44-48ce-ad55-5595c8f885b5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.641508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.642109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.642393 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.642671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.643325 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.645044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.656236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1286/2576] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:55.660281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.670871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e862fda-8f42-482d-9b08-95e276efbb2b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.675436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539b022-e8e6-4878-b4ca-0dbef1009e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.676049 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539b022-e8e6-4878-b4ca-0dbef1009e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.676330 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539b022-e8e6-4878-b4ca-0dbef1009e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.676600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539b022-e8e6-4878-b4ca-0dbef1009e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.677234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539b022-e8e6-4878-b4ca-0dbef1009e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.678823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539b022-e8e6-4878-b4ca-0dbef1009e80 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.690149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1291/2577] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:55.694844 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:55.706107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0dfaff-df3d-47f3-be75-7a9ad19e8e82 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:55.713189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:55.713837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.714166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.714397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] PATH_INFO: `/v3/projects/5a4aef20e593468aa52727ddf97764f4/users/d837e1b7d6d645e395f6fa24c6c00f51/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.731229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4aef20e593468aa52727ddf97764f4, user_id=d837e1b7d6d645e395f6fa24c6c00f51, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:55.734892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:55.748288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1287/2578] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:55 2022] PUT /identity/v3/projects/5a4aef20e593468aa52727ddf97764f4/users/d837e1b7d6d645e395f6fa24c6c00f51/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:55.815904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:55.816220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:55.816494 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:55.847446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] MFA Rules not processed for user `d837e1b7d6d645e395f6fa24c6c00f51`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:55.848686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:55.881789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:55.921989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1292/2579] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:55.926130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.008572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472e43f-890f-48a6-ae83-6c1dd9d48d37 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.014062 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.014954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.015244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.015522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.016192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.018092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.041716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:56.056828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1288/2580] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:55 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:56.061192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.073833 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f9ffd-1752-489f-8c2d-3f999afbd1d4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.140279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.141012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.141425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.141826 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.142583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.144939 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.168370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1293/2581] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:56 2022] POST /identity/v3/users => generated 297 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:56.172394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.183824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a6bcf-d66b-48aa-b9ee-19acb97978b2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.188702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.189306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.189601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.189874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.190522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.192214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.203808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1289/2582] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:56.208297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.219757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a3ac4-a188-4ebf-aac7-4b054520b869 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.227621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.228434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.228827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.229218 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] PATH_INFO: `/v3/projects/e1c0fe85d6494d70b1a9651a0f301d4e/users/88bc99817be2458b88c06156178a1439/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.256268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c0fe85d6494d70b1a9651a0f301d4e, user_id=88bc99817be2458b88c06156178a1439, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.258942 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.274646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1294/2583] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:56 2022] PUT /identity/v3/projects/e1c0fe85d6494d70b1a9651a0f301d4e/users/88bc99817be2458b88c06156178a1439/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:56.279724 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.292694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373c6c8-000f-4463-86cb-1ac44660b304 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.356460 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12685399-ef3f-4cdf-a7c3-6bfc63c06aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.357069 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12685399-ef3f-4cdf-a7c3-6bfc63c06aae admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.357535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12685399-ef3f-4cdf-a7c3-6bfc63c06aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.357893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12685399-ef3f-4cdf-a7c3-6bfc63c06aae admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.358511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12685399-ef3f-4cdf-a7c3-6bfc63c06aae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.360130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12685399-ef3f-4cdf-a7c3-6bfc63c06aae admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.370095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1290/2584] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:12:56.375607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.386140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0124a405-425a-4269-b7e4-4aec6a01eee5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.391867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.392522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.392804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.393079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] PATH_INFO: `/v3/projects/e1c0fe85d6494d70b1a9651a0f301d4e/users/88bc99817be2458b88c06156178a1439/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.394999 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1c0fe85d6494d70b1a9651a0f301d4e, user_id=88bc99817be2458b88c06156178a1439, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.397719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.409065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1295/2585] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:56 2022] PUT /identity/v3/projects/e1c0fe85d6494d70b1a9651a0f301d4e/users/88bc99817be2458b88c06156178a1439/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:56.419222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.419513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.419798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.447834 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] MFA Rules not processed for user `88bc99817be2458b88c06156178a1439`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:56.449077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:56.518765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1291/2586] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:56 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:56.530632 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.532570 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:56.592549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9655d-a7b6-404d-a367-f2b330f395c2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.596327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.596988 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.597253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.597559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.606539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.608893 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.618423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1296/2587] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:56 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:56.694650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:56.704519 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf44f3b-2855-4586-96a3-47be24f71952 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:56.708213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:56.708811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:56.709099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:56.709375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:56.774759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:56.776236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:56.786366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1292/2588] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:12:57.062470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:57.097975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:57.133982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495864a-5135-4bc5-afe3-84eab10cf92b service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:57.139424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:57.140233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:57.140589 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:57.140969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] PATH_INFO: `/v3/users/d837e1b7d6d645e395f6fa24c6c00f51` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:57.142241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=d837e1b7d6d645e395f6fa24c6c00f51)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:57.144537 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:57.197643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd837e1b7d6d645e395f6fa24c6c00f51'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.209020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd837e1b7d6d645e395f6fa24c6c00f51'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.212274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:57.213702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:57.218706 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd837e1b7d6d645e395f6fa24c6c00f51'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.222137 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:57.233296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd837e1b7d6d645e395f6fa24c6c00f51'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.234136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-bcd2d478-27f9-4d90-959e-58bd3aa68bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd837e1b7d6d645e395f6fa24c6c00f51'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.240450 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1297/2589] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:57 2022] DELETE /identity/v3/users/d837e1b7d6d645e395f6fa24c6c00f51 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:57.244870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:57.245157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:57.245439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:57.265426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:57.266638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:57.275709 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad98bb-603f-41a8-b63b-1c0825c7f8a3 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:57.349160 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:57.349769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:57.350047 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:57.350318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:57.352114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:57.352310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1298/2590] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:57.362091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:57.363500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:57.366289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:57.367736 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:57.378093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f663ea-2835-41c8-8ef2-74194cf05905 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:57.378668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1293/2591] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:12:57.382546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:57.383169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:57.383596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:57.383868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:57.393976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:57.395423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:57.408460 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1299/2592] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:12:57.886007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:57.887602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:57.901932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e74f-48f1-45d8-9103-99b5608ea92b service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:57.907919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:57.908605 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:57.908890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:57.909167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] PATH_INFO: `/v3/projects/5a4aef20e593468aa52727ddf97764f4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:57.910326 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4aef20e593468aa52727ddf97764f4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:57.912366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:57.965470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4aef20e593468aa52727ddf97764f4', 'request_id': 'req-60826f23-db5c-44aa-9f7b-1269aabdbc0d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.974777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4aef20e593468aa52727ddf97764f4', 'request_id': 'req-60826f23-db5c-44aa-9f7b-1269aabdbc0d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.983600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4aef20e593468aa52727ddf97764f4', 'request_id': 'req-60826f23-db5c-44aa-9f7b-1269aabdbc0d'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.996334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] The token cache is being invalidate because project 5a4aef20e593468aa52727ddf97764f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:57.996647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.997305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:57.998974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1294/2593] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:57 2022] DELETE /identity/v3/projects/5a4aef20e593468aa52727ddf97764f4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:58.002919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:58.013619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fab97-72fb-44fe-a196-5b48b31a322b service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:58.053010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:58.082483 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:58.083171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:58.083403 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:58.083683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] PATH_INFO: `/v3/users/88bc99817be2458b88c06156178a1439` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:58.084904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=88bc99817be2458b88c06156178a1439)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:58.086879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:58.145447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88bc99817be2458b88c06156178a1439'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.155582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88bc99817be2458b88c06156178a1439'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.168543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88bc99817be2458b88c06156178a1439'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.187868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88bc99817be2458b88c06156178a1439'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.188660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5298a170-0385-4706-b697-d20ea8c8bffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88bc99817be2458b88c06156178a1439'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.197683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1300/2594] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:12:58 2022] DELETE /identity/v3/users/88bc99817be2458b88c06156178a1439 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:58.485518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:58.486850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:58.503718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826f23-db5c-44aa-9f7b-1269aabdbc0d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:58.552721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:58.572782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:58.573450 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:58.573803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:58.574151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] PATH_INFO: `/v3/projects/e1c0fe85d6494d70b1a9651a0f301d4e` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:58.575408 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1c0fe85d6494d70b1a9651a0f301d4e)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:58.578387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:58.629559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1c0fe85d6494d70b1a9651a0f301d4e', 'request_id': 'req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.637515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1c0fe85d6494d70b1a9651a0f301d4e', 'request_id': 'req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.644697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1c0fe85d6494d70b1a9651a0f301d4e', 'request_id': 'req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.657368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] The token cache is being invalidate because project e1c0fe85d6494d70b1a9651a0f301d4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:12:58.657647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.658324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-bd5e3d52-1b69-4ef4-9e6e-b6945709d618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:12:58.659824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1295/2595] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:12:58 2022] DELETE /identity/v3/projects/e1c0fe85d6494d70b1a9651a0f301d4e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:12:58.664753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:58.665037 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:58.665312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:58.687338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:58.688696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:58.767485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1301/2596] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:12:58.778475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:58.786205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:58.786205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:58.813234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:58.814477 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:58.823544 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1296/2597] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:58.828139 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:58.838817 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c467a272-4dc7-4feb-9d72-e51eb99d7ec9 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:58.843928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:58.844766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:58.845023 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:58.845315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:58.845966 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:12:58.846272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:58.847890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:58.864165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1302/2598] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:12:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:12:58.868626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:58.879463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a038111-845a-413f-92e8-226d920b57c5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:58.884399 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:58.885068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:58.885344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:58.885620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:58.886274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:58.888358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:58.929726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1297/2599] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:12:58 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:12:58.933853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:58.944770 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de605ae9-327f-4d6d-a688-d2989a3e6b9a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:58.987795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:12:59.006417 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:59.007003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:59.007264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:59.007539 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:59.008277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:59.010217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:59.034834 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1303/2600] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:12:58 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:12:59.038586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:59.051616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa205c1-82da-40f5-9a72-ea6011b9e1e4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:59.056630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:59.057237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:59.057520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:59.057797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:59.058637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:59.060835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:59.073343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1298/2601] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:12:59.077631 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:59.088021 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f3fb73-47e8-4ba0-ac26-b534159b0555 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:59.093145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb135b11-b11e-4828-83f3-b959382dd17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:59.093743 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb135b11-b11e-4828-83f3-b959382dd17e admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:59.094019 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb135b11-b11e-4828-83f3-b959382dd17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:59.094289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb135b11-b11e-4828-83f3-b959382dd17e admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:59.094938 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb135b11-b11e-4828-83f3-b959382dd17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:59.096559 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb135b11-b11e-4828-83f3-b959382dd17e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:59.108316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1304/2602] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:12:59.113273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:59.124944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893f702-9c3b-4a59-8f7a-c8b6b2e95146 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:59.130513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:59.131176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:59.131453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:59.131789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] PATH_INFO: `/v3/projects/5c27f7bbe3cc4526b7c3c6c7c69890e5/users/5cc0762826a44b6fa366a73b0b13df0a/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:59.148000 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c27f7bbe3cc4526b7c3c6c7c69890e5, user_id=5cc0762826a44b6fa366a73b0b13df0a, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:59.150677 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:59.165660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1299/2603] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:12:59 2022] PUT /identity/v3/projects/5c27f7bbe3cc4526b7c3c6c7c69890e5/users/5cc0762826a44b6fa366a73b0b13df0a/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:12:59.177936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:59.178242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:59.178500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:59.215467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] MFA Rules not processed for user `5cc0762826a44b6fa366a73b0b13df0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:12:59.218070 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:59.305728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1305/2604] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:12:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:12:59.318666 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:12:59.321304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:12:59.388133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f503aee-eaf5-4940-a9de-60c32bf31a00 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:12:59.392140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:12:59.392940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:12:59.393379 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:12:59.393839 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:12:59.410318 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:12:59.418579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:12:59.431103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1300/2605] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:12:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:01.131362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:01.136456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:01.205125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b886a7bb-b8f7-4fc6-ab2c-0eb735dfc818 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:01.209703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:01.210332 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.210613 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.210889 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] PATH_INFO: `/v3/users/714872990c384663bcd564cd5b9ae432` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.212066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] RBAC: Authorizing `identity:delete_user(user_id=714872990c384663bcd564cd5b9ae432)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:01.213981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:01.263738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714872990c384663bcd564cd5b9ae432'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.269901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:01.273480 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714872990c384663bcd564cd5b9ae432'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.284471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714872990c384663bcd564cd5b9ae432'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.304546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714872990c384663bcd564cd5b9ae432'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.305293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8e5541e5-70bd-49ca-a49f-99c18f33849f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714872990c384663bcd564cd5b9ae432'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.313538 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1306/2606] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:01 2022] DELETE /identity/v3/users/714872990c384663bcd564cd5b9ae432 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:01.507376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:01.525098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e94c1-5802-4548-8b5c-73f8eb57ec32 service glance] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:01.537667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:01.588417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:01.589043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.589320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.589593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] PATH_INFO: `/v3/projects/a25cd48f364541dbaceabdac555bb343` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.590702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25cd48f364541dbaceabdac555bb343)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:01.593475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:01.645324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25cd48f364541dbaceabdac555bb343', 'request_id': 'req-c96d486b-b35e-47d9-98fe-ada2c67255af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.653140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25cd48f364541dbaceabdac555bb343', 'request_id': 'req-c96d486b-b35e-47d9-98fe-ada2c67255af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.659583 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25cd48f364541dbaceabdac555bb343', 'request_id': 'req-c96d486b-b35e-47d9-98fe-ada2c67255af'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.671508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] The token cache is being invalidate because project a25cd48f364541dbaceabdac555bb343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:01.671796 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.672466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c96d486b-b35e-47d9-98fe-ada2c67255af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:01.673952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1301/2607] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:01 2022] DELETE /identity/v3/projects/a25cd48f364541dbaceabdac555bb343 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:01.678887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.679205 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.679478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.698924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:01.700154 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:01.771350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1307/2608] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:01.780907 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.781194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.781473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.801936 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:01.803146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:01.811046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1302/2609] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:01.815376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:01.825002 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab323075-d12d-447c-a024-9af06d5a4270 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:01.830582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:01.831254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.831650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.831927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.832808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:01.833350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:01.835767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:01.853608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1308/2610] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:01.858078 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:01.867360 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af36770-4538-43c5-a03a-22b7894c9c37 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:01.871987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:01.872674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.872986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.873348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.873986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:01.875880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:01.903395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1303/2611] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:01 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:01.908236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:01.918026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8dc4-ead3-4b66-a389-eba970fe05e2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:01.985164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:01.985905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:01.986308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:01.986696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:01.987389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:01.989387 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:02.011671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1309/2612] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:01 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:02.016658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:02.031125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a4a6f-4770-4476-9696-cc8eb6f68e4b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:02.035813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:02.036456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:02.036742 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:02.037002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:02.037655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:02.039233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:02.050601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1304/2613] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:02.054805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:02.066678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a7205-9b54-4b84-b51f-02d0b230c1df admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:02.071772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa86cb1-4002-4f00-bd15-4900654bc5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:02.072398 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa86cb1-4002-4f00-bd15-4900654bc5ad admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:02.072678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa86cb1-4002-4f00-bd15-4900654bc5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:02.072952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa86cb1-4002-4f00-bd15-4900654bc5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:02.073622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa86cb1-4002-4f00-bd15-4900654bc5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:02.076583 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa86cb1-4002-4f00-bd15-4900654bc5ad admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:02.089159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1310/2614] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:02.094504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:02.108183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7cbdb-ad5d-4b3d-ba35-2aa96d08d671 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:02.113564 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:02.114250 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:02.114624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:02.114903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] PATH_INFO: `/v3/projects/8ae1e2afc0894009aca8e419950c4703/users/4b48479848c54c059655e11a169ddaf4/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:02.132813 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae1e2afc0894009aca8e419950c4703, user_id=4b48479848c54c059655e11a169ddaf4, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:02.135471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:02.156832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1305/2615] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:02 2022] PUT /identity/v3/projects/8ae1e2afc0894009aca8e419950c4703/users/4b48479848c54c059655e11a169ddaf4/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:02.167143 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:02.167729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:02.167729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:02.199067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] MFA Rules not processed for user `4b48479848c54c059655e11a169ddaf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:02.200372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:02.232606 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:02.284746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1311/2616] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:02.298500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:02.300503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:02.349648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:02.390102 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7534b3-7262-4133-8392-a12e3f3b7dda admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:02.396259 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46b63b-43ce-4559-b02a-ee4fc87afc97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:02.396820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b63b-43ce-4559-b02a-ee4fc87afc97 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:02.397125 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b63b-43ce-4559-b02a-ee4fc87afc97 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:02.397397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46b63b-43ce-4559-b02a-ee4fc87afc97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:02.407772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b63b-43ce-4559-b02a-ee4fc87afc97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:02.409763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46b63b-43ce-4559-b02a-ee4fc87afc97 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:02.424854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1306/2617] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:09.789969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:09.855926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801244-fd33-4a8c-8e8c-7018c71a4b9d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:09.861091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:09.861735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:09.862013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:09.862283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] PATH_INFO: `/v3/users/4b48479848c54c059655e11a169ddaf4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:09.863399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b48479848c54c059655e11a169ddaf4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:09.865325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:09.917149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b48479848c54c059655e11a169ddaf4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:09.926030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b48479848c54c059655e11a169ddaf4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:09.935899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b48479848c54c059655e11a169ddaf4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:09.949344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b48479848c54c059655e11a169ddaf4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:09.950041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b48479848c54c059655e11a169ddaf4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:09.956634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1312/2618] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:09 2022] DELETE /identity/v3/users/4b48479848c54c059655e11a169ddaf4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:09.961184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:09.961885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:09.961885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:09.984970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:09.986188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:10.059828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1307/2619] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:10.066478 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:10.067961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:10.121860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:10.136255 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263fe9d-f353-4d91-b93d-7bd91a07b863 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:10.141277 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059b25a-b4ac-4c45-925b-44c5053a4929 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:10.141881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059b25a-b4ac-4c45-925b-44c5053a4929 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.142184 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059b25a-b4ac-4c45-925b-44c5053a4929 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.142452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059b25a-b4ac-4c45-925b-44c5053a4929 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.151907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059b25a-b4ac-4c45-925b-44c5053a4929 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:10.153397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059b25a-b4ac-4c45-925b-44c5053a4929 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:10.165213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1313/2620] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:10.448314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:10.450588 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:10.469594 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc470776-63a3-4dcc-89db-b0a5e57d857e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:10.483900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:10.484587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.484868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.485142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] PATH_INFO: `/v3/projects/8ae1e2afc0894009aca8e419950c4703` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.486298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae1e2afc0894009aca8e419950c4703)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:10.488238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:10.536800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae1e2afc0894009aca8e419950c4703', 'request_id': 'req-792cf488-b5e1-470b-acf9-381fe695c5e2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:10.545902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae1e2afc0894009aca8e419950c4703', 'request_id': 'req-792cf488-b5e1-470b-acf9-381fe695c5e2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:10.551578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae1e2afc0894009aca8e419950c4703', 'request_id': 'req-792cf488-b5e1-470b-acf9-381fe695c5e2'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:10.562675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] The token cache is being invalidate because project 8ae1e2afc0894009aca8e419950c4703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:10.562963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:10.563609 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-792cf488-b5e1-470b-acf9-381fe695c5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:10.565130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1308/2621] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:10 2022] DELETE /identity/v3/projects/8ae1e2afc0894009aca8e419950c4703 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:10.569855 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.570214 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.570513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.589792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:10.590997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:10.652459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1314/2622] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:10.661385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.661946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.662403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.810887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:10.812260 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:10.820164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1309/2623] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:10.824566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:10.834919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea0385-87b4-4234-8ad6-01e8be025e76 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:10.839929 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:10.840573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.840856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.841137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.842479 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:10.842807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:10.844453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:10.856282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1315/2624] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:10.860850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:10.869958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a8e1c-4a9c-4557-a739-80412fb745d5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:10.874911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:10.875512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.875790 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.876103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.876747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:10.878629 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:10.909247 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1310/2625] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:10 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:10.913712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:10.925551 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81594cc2-5ee0-4c51-91c8-9fffc22caf2c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:10.991732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:10.992383 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:10.992661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:10.992936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:10.993638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:10.995699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.022921 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1316/2626] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:10 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:11.023735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.035830 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432ee54-4b02-44f1-b718-c9438316c7cb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.041038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.041625 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.041908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.042191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.042826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.044459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.054986 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1311/2627] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:11.059280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.069696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4c12c-7d30-4c2f-8275-e33400867745 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.074601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253191c-135e-44ba-ab73-a594bd3bd134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.075211 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253191c-135e-44ba-ab73-a594bd3bd134 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.075492 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253191c-135e-44ba-ab73-a594bd3bd134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.075766 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253191c-135e-44ba-ab73-a594bd3bd134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.076968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253191c-135e-44ba-ab73-a594bd3bd134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.078764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253191c-135e-44ba-ab73-a594bd3bd134 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.090012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1317/2628] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:11.095237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.105202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b036-71fc-409f-b774-14a8b13287a1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.109967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.110637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.110848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.111117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] PATH_INFO: `/v3/projects/2c650a52e1ab48ca9a50e0ee200583df/users/3e3de522b4794a7e9b5c2494d52d9c72/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.120072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:11.129031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c650a52e1ab48ca9a50e0ee200583df, user_id=3e3de522b4794a7e9b5c2494d52d9c72, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.132700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.147103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1312/2629] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:11 2022] PUT /identity/v3/projects/2c650a52e1ab48ca9a50e0ee200583df/users/3e3de522b4794a7e9b5c2494d52d9c72/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:11.159543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.159828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.160137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.189456 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] MFA Rules not processed for user `3e3de522b4794a7e9b5c2494d52d9c72`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:11.190668 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:11.232776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:11.256100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1318/2630] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:11.260773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.332053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e2b19-cac0-40d7-a30c-ca5d23e842c3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.347290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.347931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.348242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.348547 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.349208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.351151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.381606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1313/2631] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:11 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:11.385635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.397368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde186a5-7b06-499d-8082-657ce26d1782 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.463993 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.464630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.464910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.465185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.465824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.467944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.489708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1319/2632] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:11 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:11.493740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.504752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c65d-87c3-4ef0-90ec-12678ea45606 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.509439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.510394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.510694 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.510961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.511630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.513246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.524294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1314/2633] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:11.528486 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.539397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade7e46-9524-43b5-87fb-cdc5e303442c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.544032 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.544681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.544987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.545259 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] PATH_INFO: `/v3/projects/30042f25b5874c46b67c41849cc480fa/users/acf5bffdaabc48b3a2d4b16f3add4ed1/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.562322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=30042f25b5874c46b67c41849cc480fa, user_id=acf5bffdaabc48b3a2d4b16f3add4ed1, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.565129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.580627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1320/2634] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:11 2022] PUT /identity/v3/projects/30042f25b5874c46b67c41849cc480fa/users/acf5bffdaabc48b3a2d4b16f3add4ed1/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:11.584613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.594866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4a0c7-98cc-47ea-bddb-ad033759f6fa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.658372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f284ab2-bb4b-41c1-8b3d-c3bdef731743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.659057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284ab2-bb4b-41c1-8b3d-c3bdef731743 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.659336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284ab2-bb4b-41c1-8b3d-c3bdef731743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.659606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f284ab2-bb4b-41c1-8b3d-c3bdef731743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.660282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f284ab2-bb4b-41c1-8b3d-c3bdef731743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.661870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f284ab2-bb4b-41c1-8b3d-c3bdef731743 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.672910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1315/2635] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:11.677544 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.688724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f88db-1f8b-49e0-b1dc-d48fc13840bf admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.693469 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.694124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.694405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.694680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] PATH_INFO: `/v3/projects/30042f25b5874c46b67c41849cc480fa/users/acf5bffdaabc48b3a2d4b16f3add4ed1/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.696684 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=30042f25b5874c46b67c41849cc480fa, user_id=acf5bffdaabc48b3a2d4b16f3add4ed1, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.699266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.709981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1321/2636] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:11 2022] PUT /identity/v3/projects/30042f25b5874c46b67c41849cc480fa/users/acf5bffdaabc48b3a2d4b16f3add4ed1/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:11.720428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.720795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.721265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.750238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] MFA Rules not processed for user `acf5bffdaabc48b3a2d4b16f3add4ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:11.751442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:11.798484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:11.825623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1316/2637] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:11.838457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:11.840190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:11.897350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9b5db-2f32-464b-9e62-f3f392c06aec admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:11.901268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:11.901863 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:11.902140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:11.902412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:11.911547 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:11.912983 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:11.922753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1322/2638] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:11.997189 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:12.007860 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c3870-4503-4200-bfee-69284a0b1b86 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:12.011785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7f81d-8612-4af5-b321-016362faa87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:12.012437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f81d-8612-4af5-b321-016362faa87f service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:12.012719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f81d-8612-4af5-b321-016362faa87f service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:12.012994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7f81d-8612-4af5-b321-016362faa87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:12.072327 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7f81d-8612-4af5-b321-016362faa87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:12.073780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc7f81d-8612-4af5-b321-016362faa87f service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:12.083014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1317/2639] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:18.321325 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:18.323761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:18.377644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:18.487484 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9fe53-a965-4b37-8f62-36d554290eeb service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:18.492669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:18.493792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:18.493792 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:18.494257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] PATH_INFO: `/v3/users/5cc0762826a44b6fa366a73b0b13df0a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:18.496242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc0762826a44b6fa366a73b0b13df0a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:18.505697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:18.577886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc0762826a44b6fa366a73b0b13df0a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:18.585774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc0762826a44b6fa366a73b0b13df0a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:18.593846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc0762826a44b6fa366a73b0b13df0a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:18.607746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc0762826a44b6fa366a73b0b13df0a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:18.608680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc0762826a44b6fa366a73b0b13df0a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:18.614634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1323/2640] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:18 2022] DELETE /identity/v3/users/5cc0762826a44b6fa366a73b0b13df0a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:18.619216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:18.619719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:18.619838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:18.649499 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:18.650725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:18.712441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1318/2641] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:18.723701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:18.725369 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:18.804630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:18.821970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e695b9-ba36-49fa-8c03-d923365c28b5 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:18.825840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74048e9-3502-45f8-adda-93134e8befb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:18.826585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74048e9-3502-45f8-adda-93134e8befb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:18.826948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74048e9-3502-45f8-adda-93134e8befb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:18.827284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74048e9-3502-45f8-adda-93134e8befb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:18.836580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74048e9-3502-45f8-adda-93134e8befb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:18.838113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74048e9-3502-45f8-adda-93134e8befb0 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:18.848995 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1324/2642] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:19.175192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.176962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:19.191404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c648a6e-315c-4cef-95fa-7b8ed27c90af None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.196493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.197256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.197634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.197966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] PATH_INFO: `/v3/projects/5c27f7bbe3cc4526b7c3c6c7c69890e5` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.199372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c27f7bbe3cc4526b7c3c6c7c69890e5)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:19.206965 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:19.272703 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c27f7bbe3cc4526b7c3c6c7c69890e5', 'request_id': 'req-072555a2-57ff-4e1f-b166-b42f5bfd4409'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:19.292880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c27f7bbe3cc4526b7c3c6c7c69890e5', 'request_id': 'req-072555a2-57ff-4e1f-b166-b42f5bfd4409'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:19.302744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c27f7bbe3cc4526b7c3c6c7c69890e5', 'request_id': 'req-072555a2-57ff-4e1f-b166-b42f5bfd4409'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:19.313032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:19.318913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] The token cache is being invalidate because project 5c27f7bbe3cc4526b7c3c6c7c69890e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:19.320387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:19.321463 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-072555a2-57ff-4e1f-b166-b42f5bfd4409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:19.323962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1319/2643] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:19 2022] DELETE /identity/v3/projects/5c27f7bbe3cc4526b7c3c6c7c69890e5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:19.328733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.329239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.329355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.396372 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:19.397590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:19.479928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1325/2644] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:19.488938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.489254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.489522 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.509383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:19.510576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:19.519028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1320/2645] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:19.523317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.534137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b0c0e6-49f6-4d8d-9d3a-1c248a12e281 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.539505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.540138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.540958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.540958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.541328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:19.541630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:19.543221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:19.555847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1326/2646] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:19.560721 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.571315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b108463-0146-4ff5-8ca4-03fef689d22e None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.576748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.577484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.577898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.578172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.578810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:19.580731 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:19.629747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1321/2647] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:19 2022] POST /identity/v3/projects => generated 359 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:19.631579 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.647837 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947228c0-1827-4e7e-adb1-d98e41b31651 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.760546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.761552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.762043 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.762543 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.764058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:19.767958 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:19.830635 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1327/2648] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:19 2022] POST /identity/v3/users => generated 280 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:19.836308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.854135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0488f01-bbbf-459f-8a72-0e5e2f66ba0f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.861931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.863091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.863770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.864254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.865162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:19.867648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:19.888633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1322/2649] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:19.890756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.908662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3dc521-3b25-41ae-a322-53842ea7551c admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.930503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.931485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.932273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.932557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.932832 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:19.933599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:19.934081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:19.935524 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:19.954960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1328/2650] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:19.956248 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:19.968389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185a45f-9ca4-4fe0-98f1-2b343d8472cd admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:19.975689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe82718-d32f-4c0d-91a2-6a925daab309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:19.976853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe82718-d32f-4c0d-91a2-6a925daab309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:19.977776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe82718-d32f-4c0d-91a2-6a925daab309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:19.978358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe82718-d32f-4c0d-91a2-6a925daab309 admin admin] PATH_INFO: `/v3/projects/1ac3a8cc42454de8ba69ca124241c7da/users/4062e680676a48e499b64082a37c38d9/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.010656 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe82718-d32f-4c0d-91a2-6a925daab309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ac3a8cc42454de8ba69ca124241c7da, user_id=4062e680676a48e499b64082a37c38d9, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:20.016628 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe82718-d32f-4c0d-91a2-6a925daab309 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:20.039024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:20.040602 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1a28-0081-4716-ac9e-07bd8c165388 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:20.064787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1329/2651] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:19 2022] PUT /identity/v3/projects/1ac3a8cc42454de8ba69ca124241c7da/users/4062e680676a48e499b64082a37c38d9/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:20.079653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:20.081263 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:20.082284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.174667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:20.175288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:20.175578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:20.177456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.180744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:20.218713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] MFA Rules not processed for user `4062e680676a48e499b64082a37c38d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:20.221066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:20.304563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:20.306245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:20.319597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1323/2652] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:20.335702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1330/2653] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:20.380094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:20.381618 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:20.420262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:20.458091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b1e952-105d-4444-a808-1bfb9a8e94c6 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:20.461893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfcb727-ae7f-437d-87d6-6a5df2275c91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:20.462531 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcb727-ae7f-437d-87d6-6a5df2275c91 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:20.462808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcb727-ae7f-437d-87d6-6a5df2275c91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:20.463080 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfcb727-ae7f-437d-87d6-6a5df2275c91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.472226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcb727-ae7f-437d-87d6-6a5df2275c91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:20.473772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfcb727-ae7f-437d-87d6-6a5df2275c91 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:20.483972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1324/2654] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:20.501435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:20.582445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ba113-5af7-48d5-abf0-0b0dad89fca4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:20.588169 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:20.588822 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:20.589298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:20.589436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] PATH_INFO: `/v3/users/4062e680676a48e499b64082a37c38d9` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.590642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4062e680676a48e499b64082a37c38d9)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:20.593722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:20.652952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4062e680676a48e499b64082a37c38d9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:20.661823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4062e680676a48e499b64082a37c38d9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:20.670493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4062e680676a48e499b64082a37c38d9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:20.685679 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4062e680676a48e499b64082a37c38d9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:20.686425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4062e680676a48e499b64082a37c38d9'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:20.692976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1331/2655] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:20 2022] DELETE /identity/v3/users/4062e680676a48e499b64082a37c38d9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:20.697626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:20.698457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:20.698738 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.718075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:20.719283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:20.790176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1325/2656] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:20.811295 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:20.814045 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:20.835466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:20.896593 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053d57-cd66-4f3a-872c-a55af37582c7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:20.900391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f2081-1489-45b0-b670-050bec211daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:20.901025 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2081-1489-45b0-b670-050bec211daf service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:20.901304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2081-1489-45b0-b670-050bec211daf service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:20.901576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f2081-1489-45b0-b670-050bec211daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:20.915338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f2081-1489-45b0-b670-050bec211daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:20.916818 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f2081-1489-45b0-b670-050bec211daf service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:20.929767 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1332/2657] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:21.324995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:21.327740 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:21.344719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52aca5d-bfef-43b5-b474-b0ff0a5e7058 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:21.350554 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:21.351230 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:21.351515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:21.351798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] PATH_INFO: `/v3/users/3e3de522b4794a7e9b5c2494d52d9c72` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:21.353020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e3de522b4794a7e9b5c2494d52d9c72)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:21.354961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:21.415882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3de522b4794a7e9b5c2494d52d9c72'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:21.416942 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3de522b4794a7e9b5c2494d52d9c72'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:21.428600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3de522b4794a7e9b5c2494d52d9c72'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:21.445801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3de522b4794a7e9b5c2494d52d9c72'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:21.459908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3de522b4794a7e9b5c2494d52d9c72'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:21.468106 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1326/2658] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:21 2022] DELETE /identity/v3/users/3e3de522b4794a7e9b5c2494d52d9c72 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:21.472100 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:21.473054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:21.473345 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:21.499581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:21.501206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:21.585290 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1333/2659] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:21.593010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:21.594521 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:21.688788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:21.708447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036f499-4907-4948-abaa-418403cf671f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:21.716378 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:21.717124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:21.717437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:21.718853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:21.745886 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:21.748358 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:21.761063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1327/2660] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:21.976616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:21.980586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:22.004970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce43255-40a9-49f2-ae72-094ab20989d7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:22.022794 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:22.023727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:22.025041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:22.025554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] PATH_INFO: `/v3/projects/1ac3a8cc42454de8ba69ca124241c7da` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:22.030452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ac3a8cc42454de8ba69ca124241c7da)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:22.033839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:22.117352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ac3a8cc42454de8ba69ca124241c7da', 'request_id': 'req-b9f6b62d-d54a-4986-8184-8c19825a725a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.128660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ac3a8cc42454de8ba69ca124241c7da', 'request_id': 'req-b9f6b62d-d54a-4986-8184-8c19825a725a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.138986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ac3a8cc42454de8ba69ca124241c7da', 'request_id': 'req-b9f6b62d-d54a-4986-8184-8c19825a725a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.162339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] The token cache is being invalidate because project 1ac3a8cc42454de8ba69ca124241c7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:22.162870 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.164056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.172854 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1334/2661] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:21 2022] DELETE /identity/v3/projects/1ac3a8cc42454de8ba69ca124241c7da => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:22.356163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:22.365891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:22.391530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea6294-d910-4793-807d-64bc0d863c3a service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:22.488541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:22.489372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:22.490008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:22.490299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] PATH_INFO: `/v3/projects/2c650a52e1ab48ca9a50e0ee200583df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:22.491757 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c650a52e1ab48ca9a50e0ee200583df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:22.494489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:22.563461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c650a52e1ab48ca9a50e0ee200583df', 'request_id': 'req-58b473a1-f1fe-4215-9190-b3933424fec0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.578590 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c650a52e1ab48ca9a50e0ee200583df', 'request_id': 'req-58b473a1-f1fe-4215-9190-b3933424fec0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.578819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c650a52e1ab48ca9a50e0ee200583df', 'request_id': 'req-58b473a1-f1fe-4215-9190-b3933424fec0'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.590139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] The token cache is being invalidate because project 2c650a52e1ab48ca9a50e0ee200583df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:22.590432 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.591155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.592898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1328/2662] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:22 2022] DELETE /identity/v3/projects/2c650a52e1ab48ca9a50e0ee200583df => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:22.596943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:22.598462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:22.616385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6b62d-d54a-4986-8184-8c19825a725a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:22.661423 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:22.690358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:22.691160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:22.691445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:22.691723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] PATH_INFO: `/v3/users/acf5bffdaabc48b3a2d4b16f3add4ed1` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:22.692941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf5bffdaabc48b3a2d4b16f3add4ed1)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:22.694853 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:22.748825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf5bffdaabc48b3a2d4b16f3add4ed1'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.757221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf5bffdaabc48b3a2d4b16f3add4ed1'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.766558 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf5bffdaabc48b3a2d4b16f3add4ed1'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.788389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf5bffdaabc48b3a2d4b16f3add4ed1'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.789195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ff487e4e-c9cf-440a-8fb9-02b8cf23d649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf5bffdaabc48b3a2d4b16f3add4ed1'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:22.797618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1335/2663] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:22 2022] DELETE /identity/v3/users/acf5bffdaabc48b3a2d4b16f3add4ed1 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:23.123574 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.125188 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:23.138156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b473a1-f1fe-4215-9190-b3933424fec0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.228217 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.228853 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.229181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.229582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] PATH_INFO: `/v3/projects/30042f25b5874c46b67c41849cc480fa` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.230905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30042f25b5874c46b67c41849cc480fa)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.234385 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.260842 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:23.297095 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30042f25b5874c46b67c41849cc480fa', 'request_id': 'req-0cf6fa8e-04ca-4d77-9952-42abf2e49580'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:23.306288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30042f25b5874c46b67c41849cc480fa', 'request_id': 'req-0cf6fa8e-04ca-4d77-9952-42abf2e49580'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:23.313673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30042f25b5874c46b67c41849cc480fa', 'request_id': 'req-0cf6fa8e-04ca-4d77-9952-42abf2e49580'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:23.325841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] The token cache is being invalidate because project 30042f25b5874c46b67c41849cc480fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:23.326131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:23.326788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-0cf6fa8e-04ca-4d77-9952-42abf2e49580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:23.328795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1329/2664] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:23 2022] DELETE /identity/v3/projects/30042f25b5874c46b67c41849cc480fa => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:23.333651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.333936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.334210 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.361008 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:23.362271 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:23.445636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1336/2665] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:23.455951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.456528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.456650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.489475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:23.490551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:23.501665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1330/2666] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:23.505987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.516897 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730fe82-c3a0-4946-aa59-8b5fb38525e8 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.521682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.522382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.522675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.524289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.524289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:23.524289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.525753 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.544898 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1337/2667] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:23.546313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.556301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ba212-09fd-4ea4-95f8-645dc6a02b42 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.561654 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.562274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.562555 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.562826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.563472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.565685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.610541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1331/2668] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:23 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:23.616827 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.633493 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b133575-9c55-46b8-8a0b-d64419158c58 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.690377 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:23.716860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.726548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.727024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.727425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.730586 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.735233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.811730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1338/2669] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:23 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:13:23.811833 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.817771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cd2e8-fdf0-4cf0-adca-0b2fab314beb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.822882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.824042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.824042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.824256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.825097 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.827212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.841836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1332/2670] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:23.843587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.853978 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac275fd-cf1b-434e-bb10-90c048660b25 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.859365 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77787d2f-6aee-4f0e-8272-26064a646df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.860046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77787d2f-6aee-4f0e-8272-26064a646df4 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.860331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77787d2f-6aee-4f0e-8272-26064a646df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.860627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77787d2f-6aee-4f0e-8272-26064a646df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.861599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77787d2f-6aee-4f0e-8272-26064a646df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.863386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77787d2f-6aee-4f0e-8272-26064a646df4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.880102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1339/2671] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:23.881471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:23.894940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86397095-8dee-4bb0-8d82-2ffaf988b0e8 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:23.902699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:23.904924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.905232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.905657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] PATH_INFO: `/v3/projects/4854f79390fe45a083f74beb760bfc77/users/fffa46008900469eaa0a5435a6c329ef/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.924114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4854f79390fe45a083f74beb760bfc77, user_id=fffa46008900469eaa0a5435a6c329ef, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:23.927094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:23.942896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1333/2672] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:23 2022] PUT /identity/v3/projects/4854f79390fe45a083f74beb760bfc77/users/fffa46008900469eaa0a5435a6c329ef/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:23.955998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:23.956405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:23.956678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:23.990711 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] MFA Rules not processed for user `fffa46008900469eaa0a5435a6c329ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:23.991937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:24.060697 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1340/2673] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:24.065371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.148462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165aa228-9c19-41ce-a873-c1f2bf6f65c3 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.153887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.154523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.154808 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.155079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.155718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.157699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.195849 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1334/2674] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:24 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:24.199650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.212575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926c61ff-1d1e-4810-b514-efed9a541a68 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.273828 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.274434 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.274717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.274990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.275637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.277585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.299922 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1341/2675] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:24 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:24.304074 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.315266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc70867f-1fec-40ec-be7a-7067a1761f98 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.319878 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.320498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.320779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.321052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.321690 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.323276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.334468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1335/2676] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:24.338491 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.349191 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e85a75-7091-4900-a859-e8d4f951f484 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.354220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab26f15-3e1e-4599-b3b9-1a7620aa8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.354910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab26f15-3e1e-4599-b3b9-1a7620aa8435 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.355225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab26f15-3e1e-4599-b3b9-1a7620aa8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.355501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab26f15-3e1e-4599-b3b9-1a7620aa8435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.356176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab26f15-3e1e-4599-b3b9-1a7620aa8435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.358556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab26f15-3e1e-4599-b3b9-1a7620aa8435 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.369363 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1342/2677] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:24.374908 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.385617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed492c7-c533-4446-b2a4-e691aa737337 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.391025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.391676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.391977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.392285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] PATH_INFO: `/v3/projects/88cf3d696f7144dbb19806859d0eaf0d/users/3f0e462d95aa4328980a37cd219f8ad7/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.412457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=88cf3d696f7144dbb19806859d0eaf0d, user_id=3f0e462d95aa4328980a37cd219f8ad7, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.415157 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.417559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:24.434507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1336/2678] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:24 2022] PUT /identity/v3/projects/88cf3d696f7144dbb19806859d0eaf0d/users/3f0e462d95aa4328980a37cd219f8ad7/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:24.446582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.447440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.447931 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.480041 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] MFA Rules not processed for user `3f0e462d95aa4328980a37cd219f8ad7`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:24.482179 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:24.526449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:24.589474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1343/2679] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:24.594076 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.623129 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.625297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:24.675241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61891e89-70df-47ba-b767-9d89abd1f08a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.680213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.680926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.681274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.681657 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.682367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.684405 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.717107 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1337/2680] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:24 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:24.721452 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.736501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebe2022-6845-4b19-a55a-372e8df9b305 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.790083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0a9f3-5df1-42b1-a08a-7cc192c9faa6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.796782 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.797601 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.797879 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.798149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.798791 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.799495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.800146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.800432 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.800713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.800855 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.802509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:24.816591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.818573 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.824838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1338/2681] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:24 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:24.828689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.834062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1344/2682] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:24.838373 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2017d-0c5c-440d-a19f-cabe16c252aa admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.844541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.845143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.845418 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.845674 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.846449 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.847910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.861348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1339/2683] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:13:24.865741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.880296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c888d62-9fda-43e4-8dfa-60edc9eed4bf service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:24.885242 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:24.885900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:24.886176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:24.886625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] PATH_INFO: `/v3/projects/36ca32c70ddf49ceaae0b9f2f33329ce/users/4066e7b1553d497d8209b46ef9ee4ec0/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:24.903918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ca32c70ddf49ceaae0b9f2f33329ce, user_id=4066e7b1553d497d8209b46ef9ee4ec0, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:24.906990 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:24.928065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1345/2684] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:24 2022] PUT /identity/v3/projects/36ca32c70ddf49ceaae0b9f2f33329ce/users/4066e7b1553d497d8209b46ef9ee4ec0/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:24.931987 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:24.942330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ddd42-727f-4861-a3c6-2dac20b4bd59 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:25.002017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:25.009133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628dda72-a126-4c6d-a3d1-8a688aeaee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:25.009949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dda72-a126-4c6d-a3d1-8a688aeaee9d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:25.010231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dda72-a126-4c6d-a3d1-8a688aeaee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:25.010507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628dda72-a126-4c6d-a3d1-8a688aeaee9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:25.011159 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dda72-a126-4c6d-a3d1-8a688aeaee9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:25.012828 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628dda72-a126-4c6d-a3d1-8a688aeaee9d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:25.027208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1340/2685] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:25.029449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:25.040094 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bcdbc-02e8-4478-b640-e2247fc0e261 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:25.045508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:25.046272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:25.046439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:25.046717 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] PATH_INFO: `/v3/projects/36ca32c70ddf49ceaae0b9f2f33329ce/users/4066e7b1553d497d8209b46ef9ee4ec0/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:25.048675 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ca32c70ddf49ceaae0b9f2f33329ce, user_id=4066e7b1553d497d8209b46ef9ee4ec0, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:25.051293 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:25.053565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:25.063296 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1346/2686] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:25 2022] PUT /identity/v3/projects/36ca32c70ddf49ceaae0b9f2f33329ce/users/4066e7b1553d497d8209b46ef9ee4ec0/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:25.074558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:25.074841 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:25.075116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:25.103969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] MFA Rules not processed for user `4066e7b1553d497d8209b46ef9ee4ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:25.105214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:25.196650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1341/2687] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:25.207241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:25.209285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:25.276819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177594db-f8ce-4e49-a84a-86e0a0978e4f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:25.281667 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:25.282280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:25.282594 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:25.282868 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:25.293790 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:25.295442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:25.305297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1347/2688] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:25.783141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:25.784809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:25.850870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df053a4b-9a29-47dc-a4f1-7fe2cedf9587 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:25.854614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:25.855283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:25.855633 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:25.855974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:25.858335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:25.867529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:25.868951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:25.880997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12808204-1547-4d53-887c-28049f695a61 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:25.885145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:25.885738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:25.886016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:25.886285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:25.925255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:25.926695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:25.937784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1342/2689] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:25.945471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:25.946911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:25.956629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1348/2690] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:26.151209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:26.161301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f2b8c-c108-40a2-9420-973d4d17056b service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:26.165308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:26.166058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:26.166357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:26.166589 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:26.168524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:26.181617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:26.183038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:26.195196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1343/2691] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:30.225150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:30.236786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b43e8-cb24-44e6-902d-4469962286fa service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:30.241504 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:30.242294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:30.242754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:30.243208 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:30.374257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:30.376436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:30.395954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1349/2692] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:31.527254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:31.529142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:31.545581 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e4a94-3e4e-4937-b77e-1397ca79ffee service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:31.551218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:31.551932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:31.552486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:31.554124 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] PATH_INFO: `/v3/users/81d937b8a6c842819458b6e3c23f80fa` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:31.555348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d937b8a6c842819458b6e3c23f80fa)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:31.562275 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:31.580335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:31.630118 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d937b8a6c842819458b6e3c23f80fa'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:31.631021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d937b8a6c842819458b6e3c23f80fa'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:31.641591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d937b8a6c842819458b6e3c23f80fa'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:31.655465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d937b8a6c842819458b6e3c23f80fa'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:31.662769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ce715893-23a1-482a-9538-7b7f6d8f49cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d937b8a6c842819458b6e3c23f80fa'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:31.669502 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1344/2693] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:31 2022] DELETE /identity/v3/users/81d937b8a6c842819458b6e3c23f80fa => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:32.050883 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.064497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53656154-8135-460b-a7e8-c938f15f7c68 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.104054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:32.137038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.137734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.138284 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.138413 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] PATH_INFO: `/v3/projects/00b397afc10243bc9b93548873add6ef` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.139604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] RBAC: Authorizing `identity:delete_project(project_id=00b397afc10243bc9b93548873add6ef)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.141625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.190204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00b397afc10243bc9b93548873add6ef', 'request_id': 'req-68b7154f-47de-45c9-a4d4-2a51bf6d715a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:32.196431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00b397afc10243bc9b93548873add6ef', 'request_id': 'req-68b7154f-47de-45c9-a4d4-2a51bf6d715a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:32.202223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00b397afc10243bc9b93548873add6ef', 'request_id': 'req-68b7154f-47de-45c9-a4d4-2a51bf6d715a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:32.218597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] The token cache is being invalidate because project 00b397afc10243bc9b93548873add6ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:32.218890 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:32.219546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-68b7154f-47de-45c9-a4d4-2a51bf6d715a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:32.221158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1350/2694] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:32 2022] DELETE /identity/v3/projects/00b397afc10243bc9b93548873add6ef => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:32.226221 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.226786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.227065 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.248356 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:32.249584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:32.321769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1345/2695] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:32.330760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.331352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.331788 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.353226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:32.354699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:32.367193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1351/2696] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:32.371055 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.381546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a43bd-b7b0-4370-a3c5-046638c78377 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.390089 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.390726 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.391105 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.391383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.392082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:32.392407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.394007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.408211 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1346/2697] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:13:32.412245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.423013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bdfe8d-746b-4dc8-be50-5e8b95b81f7a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.429177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.429925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.430282 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.430630 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.431353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.433427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.477845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1352/2698] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:32 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:13:32.483170 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.515229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea04d6e9-8eee-4eb4-ae3a-16eb245cb642 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.587648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.589183 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.589563 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.590091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.591580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.593874 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.619539 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1347/2699] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:32 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:32.623896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.638060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8486b410-2a00-4aff-ae55-0dfe4d3c1230 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.644541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.645315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.646193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.646487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.647155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.649007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.668060 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1353/2700] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:32.672218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.680630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:32.685415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650da63c-c04c-4ee7-b76c-dedd14eb04bf admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.692192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26009c6-a17d-43de-9e12-de97f1299d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.693365 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26009c6-a17d-43de-9e12-de97f1299d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.693984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26009c6-a17d-43de-9e12-de97f1299d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.695088 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26009c6-a17d-43de-9e12-de97f1299d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.696577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26009c6-a17d-43de-9e12-de97f1299d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.703154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26009c6-a17d-43de-9e12-de97f1299d37 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.719518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1348/2701] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:32.724574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.737618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e8e80-8d30-44e5-96ea-0193775c1aee admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:32.743576 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:32.744376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.744645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.744918 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] PATH_INFO: `/v3/projects/a1cc280f321b4896a9af2d59312f3fbe/users/3a0be5ca18164c849baa1760fb297479/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.765141 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cc280f321b4896a9af2d59312f3fbe, user_id=3a0be5ca18164c849baa1760fb297479, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:32.767830 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:32.795641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1354/2702] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:32 2022] PUT /identity/v3/projects/a1cc280f321b4896a9af2d59312f3fbe/users/3a0be5ca18164c849baa1760fb297479/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:32.806281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:32.806580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:32.806744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:32.837338 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] MFA Rules not processed for user `3a0be5ca18164c849baa1760fb297479`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:32.838695 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:32.932178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1349/2703] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:32.945251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:32.947405 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:33.039648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:33.041209 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:33.060805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:33.121308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813135cf-1652-4419-aa04-f4f9e2356803 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:33.132231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce780c39-8e6e-4ec6-b257-e59c21b27267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:33.133568 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce780c39-8e6e-4ec6-b257-e59c21b27267 service nova] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:33.134114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce780c39-8e6e-4ec6-b257-e59c21b27267 service nova] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:33.134580 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce780c39-8e6e-4ec6-b257-e59c21b27267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:33.145466 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce780c39-8e6e-4ec6-b257-e59c21b27267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:33.147287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce780c39-8e6e-4ec6-b257-e59c21b27267 service nova] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:33.155843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89482e07-29aa-459c-93e3-c8c5960aaf78 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:33.158340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1355/2704] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:33.161335 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:33.162013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:33.162283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:33.162565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] PATH_INFO: `/v3/users/fffa46008900469eaa0a5435a6c329ef` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:33.164343 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fffa46008900469eaa0a5435a6c329ef)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:33.165821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:33.252635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fffa46008900469eaa0a5435a6c329ef'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:33.253265 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fffa46008900469eaa0a5435a6c329ef'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:33.272046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fffa46008900469eaa0a5435a6c329ef'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:33.288286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fffa46008900469eaa0a5435a6c329ef'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:33.298193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fffa46008900469eaa0a5435a6c329ef'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:33.307474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1350/2705] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:33 2022] DELETE /identity/v3/users/fffa46008900469eaa0a5435a6c329ef => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:33.312713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b38054-6924-4e48-929a-aadedf9688de None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:33.313488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b38054-6924-4e48-929a-aadedf9688de None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:33.313930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b38054-6924-4e48-929a-aadedf9688de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:33.338298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-89b38054-6924-4e48-929a-aadedf9688de None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:33.339824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-89b38054-6924-4e48-929a-aadedf9688de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:33.431454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1356/2706] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:33.446542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:33.446542 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:33.451611 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:33.609185 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa6d1-ffca-4920-8464-da5f0ad72422 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:33.622792 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:33.623507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:33.624149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:33.624548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:33.653743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:33.655615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:33.697476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1351/2707] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:34.103426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b38054-6924-4e48-929a-aadedf9688de None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:34.108201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-89b38054-6924-4e48-929a-aadedf9688de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:34.125776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b38054-6924-4e48-929a-aadedf9688de None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:34.134876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:34.135862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:34.136715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:34.137683 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] PATH_INFO: `/v3/projects/4854f79390fe45a083f74beb760bfc77` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:34.139446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4854f79390fe45a083f74beb760bfc77)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:34.141420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:34.204203 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4854f79390fe45a083f74beb760bfc77', 'request_id': 'req-8f893e5c-528f-48e2-b557-cd4c545f0d39'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.212657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4854f79390fe45a083f74beb760bfc77', 'request_id': 'req-8f893e5c-528f-48e2-b557-cd4c545f0d39'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.222642 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4854f79390fe45a083f74beb760bfc77', 'request_id': 'req-8f893e5c-528f-48e2-b557-cd4c545f0d39'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.240022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] The token cache is being invalidate because project 4854f79390fe45a083f74beb760bfc77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:34.240336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.241566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.243438 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1357/2708] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:34 2022] DELETE /identity/v3/projects/4854f79390fe45a083f74beb760bfc77 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:34.248375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:34.259671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16460a-f58d-4616-8766-c70c90e5eec5 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:34.388958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:34.389619 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:34.389896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:34.390163 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] PATH_INFO: `/v3/users/3f0e462d95aa4328980a37cd219f8ad7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:34.391701 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0e462d95aa4328980a37cd219f8ad7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:34.394826 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:34.474924 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0e462d95aa4328980a37cd219f8ad7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.475798 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0e462d95aa4328980a37cd219f8ad7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.484438 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0e462d95aa4328980a37cd219f8ad7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.498382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0e462d95aa4328980a37cd219f8ad7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.506222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0e462d95aa4328980a37cd219f8ad7'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:34.514320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1352/2709] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:34 2022] DELETE /identity/v3/users/3f0e462d95aa4328980a37cd219f8ad7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:34.817647 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:34.824198 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:34.858973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f893e5c-528f-48e2-b557-cd4c545f0d39 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:34.922359 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:34.954697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:34.955262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:34.955388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:34.955658 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:34.955935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] PATH_INFO: `/v3/projects/88cf3d696f7144dbb19806859d0eaf0d` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:34.956600 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:34.957689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88cf3d696f7144dbb19806859d0eaf0d)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:34.960177 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:34.973022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:34.977504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472998a-43ab-4b7c-bac2-033d739f03dc admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:34.986038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:34.987136 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:34.987759 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:34.988305 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] PATH_INFO: `/v3/users/d8c0e96bc0584d8b9be73ae920bbb055` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:34.993008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c0e96bc0584d8b9be73ae920bbb055)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:34.995231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:35.035754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88cf3d696f7144dbb19806859d0eaf0d', 'request_id': 'req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.048997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88cf3d696f7144dbb19806859d0eaf0d', 'request_id': 'req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.057461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88cf3d696f7144dbb19806859d0eaf0d', 'request_id': 'req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.066919 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c0e96bc0584d8b9be73ae920bbb055'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.071021 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c0e96bc0584d8b9be73ae920bbb055'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.071930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] The token cache is being invalidate because project 88cf3d696f7144dbb19806859d0eaf0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:35.072253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.073252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.075680 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1358/2710] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:34 2022] DELETE /identity/v3/projects/88cf3d696f7144dbb19806859d0eaf0d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:35.079577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:35.081084 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:35.081304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c0e96bc0584d8b9be73ae920bbb055'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.097437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c0e96bc0584d8b9be73ae920bbb055'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.106847 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c0e96bc0584d8b9be73ae920bbb055'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.116186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1353/2711] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:34 2022] DELETE /identity/v3/users/d8c0e96bc0584d8b9be73ae920bbb055 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:35.174035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edb4d2-d1bf-416b-8fd6-77ccad2a9418 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:35.304109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:35.306408 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:35.306882 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:35.307376 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] PATH_INFO: `/v3/users/4066e7b1553d497d8209b46ef9ee4ec0` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:35.309044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=4066e7b1553d497d8209b46ef9ee4ec0)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:35.311703 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:35.342894 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:35.361843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:35.363435 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:35.387935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52180a1-c547-4486-a21a-be9d163e6077 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:35.392144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4066e7b1553d497d8209b46ef9ee4ec0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.394715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:35.395485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:35.395887 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:35.396572 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] PATH_INFO: `/v3/projects/42099ef23496441fab42f876b9e39efd` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:35.398334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=42099ef23496441fab42f876b9e39efd)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:35.400306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:35.401234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4066e7b1553d497d8209b46ef9ee4ec0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.411741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4066e7b1553d497d8209b46ef9ee4ec0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.445166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4066e7b1553d497d8209b46ef9ee4ec0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.446976 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4066e7b1553d497d8209b46ef9ee4ec0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.455637 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1359/2712] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:35 2022] DELETE /identity/v3/users/4066e7b1553d497d8209b46ef9ee4ec0 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:35.463846 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42099ef23496441fab42f876b9e39efd', 'request_id': 'req-4968537b-d619-4eac-9d8a-6d97b84ed3fb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.490754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42099ef23496441fab42f876b9e39efd', 'request_id': 'req-4968537b-d619-4eac-9d8a-6d97b84ed3fb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.498288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42099ef23496441fab42f876b9e39efd', 'request_id': 'req-4968537b-d619-4eac-9d8a-6d97b84ed3fb'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.512680 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] The token cache is being invalidate because project 42099ef23496441fab42f876b9e39efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:35.513258 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.513939 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.515454 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1354/2713] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:35 2022] DELETE /identity/v3/projects/42099ef23496441fab42f876b9e39efd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:35.520136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:35.521807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:35.597646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a0450-e727-494f-8773-1a9cc012f1da admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:35.602935 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:35.603585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:35.603873 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:35.604189 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] PATH_INFO: `/v3/users/ad280dac47e243db80bc149b9ac1a0c2` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:35.605419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad280dac47e243db80bc149b9ac1a0c2)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:35.607535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:35.664956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad280dac47e243db80bc149b9ac1a0c2'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.673080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad280dac47e243db80bc149b9ac1a0c2'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.683634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad280dac47e243db80bc149b9ac1a0c2'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.696808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad280dac47e243db80bc149b9ac1a0c2'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.697518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad280dac47e243db80bc149b9ac1a0c2'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:35.704549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1360/2714] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:35 2022] DELETE /identity/v3/users/ad280dac47e243db80bc149b9ac1a0c2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:35.782541 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:35.784835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:35.865100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968537b-d619-4eac-9d8a-6d97b84ed3fb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:35.868785 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:35.899874 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:35.901360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:35.930831 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:35.932622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:35.933524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:35.934462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:35.934875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] PATH_INFO: `/v3/projects/36ca32c70ddf49ceaae0b9f2f33329ce` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:35.936162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ca32c70ddf49ceaae0b9f2f33329ce)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:35.938149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:35.985316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9609ba-b0c4-4a24-8bb3-a9fd85b9436f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:35.990256 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ca32c70ddf49ceaae0b9f2f33329ce', 'request_id': 'req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.000123 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ca32c70ddf49ceaae0b9f2f33329ce', 'request_id': 'req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.008377 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ca32c70ddf49ceaae0b9f2f33329ce', 'request_id': 'req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.025483 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] The token cache is being invalidate because project 36ca32c70ddf49ceaae0b9f2f33329ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:36.025938 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.026712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ee7d0dd4-d134-4fe8-9f9f-c395e62ed3a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.029111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1355/2715] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:35 2022] DELETE /identity/v3/projects/36ca32c70ddf49ceaae0b9f2f33329ce => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:36.035800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce919d0-6479-4099-a279-f4d589b2058f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.036216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce919d0-6479-4099-a279-f4d589b2058f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.036433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce919d0-6479-4099-a279-f4d589b2058f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.050127 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.050769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.051075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.051353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] PATH_INFO: `/v3/projects/bce529b9ed10456da8a4e59d2826991f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.052705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce529b9ed10456da8a4e59d2826991f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.054608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.056997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8ce919d0-6479-4099-a279-f4d589b2058f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:36.058216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8ce919d0-6479-4099-a279-f4d589b2058f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.067014 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1356/2716] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:36.075994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33314d4c-7167-4627-abfa-183582661389 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.076371 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33314d4c-7167-4627-abfa-183582661389 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.076745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33314d4c-7167-4627-abfa-183582661389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.097608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-33314d4c-7167-4627-abfa-183582661389 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:36.098799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-33314d4c-7167-4627-abfa-183582661389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.104596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce529b9ed10456da8a4e59d2826991f', 'request_id': 'req-53e22854-4957-4ded-ac37-5f42282ab46a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.111645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce529b9ed10456da8a4e59d2826991f', 'request_id': 'req-53e22854-4957-4ded-ac37-5f42282ab46a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.119738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce529b9ed10456da8a4e59d2826991f', 'request_id': 'req-53e22854-4957-4ded-ac37-5f42282ab46a'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.134324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] The token cache is being invalidate because project bce529b9ed10456da8a4e59d2826991f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:36.134615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.135251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.136823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1361/2717] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:35 2022] DELETE /identity/v3/projects/bce529b9ed10456da8a4e59d2826991f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:36.161732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1357/2718] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:36.165982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.176728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e22854-4957-4ded-ac37-5f42282ab46a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.181643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.182286 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.182569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.182843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.183513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:36.183820 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.186297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.203971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1362/2719] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:36.205485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33314d4c-7167-4627-abfa-183582661389 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.217164 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33314d4c-7167-4627-abfa-183582661389 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.222485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.223967 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.224302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.224575 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.225245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.227455 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.261346 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.263841 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.267237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1358/2720] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:36.271869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.278584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81cff2-b50e-44ee-86a5-c6d6c7ed3657 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.285669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:36.330840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:36.351526 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b08636-5441-4906-93fc-72609e5d0cb6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.354328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.355329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.356534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.357102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] PATH_INFO: `/v3/users/3a0be5ca18164c849baa1760fb297479` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.357294 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.357926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.358210 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.358487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.359139 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.360067 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a0be5ca18164c849baa1760fb297479)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.361140 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.362522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.386727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1359/2721] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/users => generated 305 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:36.391551 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.406447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f124bf-dbc5-4707-8880-541f7a142f3b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.412493 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.413037 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.413323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.413658 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.414334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.416033 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.427947 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a0be5ca18164c849baa1760fb297479'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.431178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1360/2722] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:13:36.435495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.437937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a0be5ca18164c849baa1760fb297479'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.445269 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6743b6-568c-4b19-86af-16a02f2e5a69 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.446379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a0be5ca18164c849baa1760fb297479'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.450272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.450865 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.451144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.451414 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.452086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.453712 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.459984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a0be5ca18164c849baa1760fb297479'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.460856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-83aae2b0-e9e0-484b-8835-f61bd53b126d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a0be5ca18164c849baa1760fb297479'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:36.464464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1361/2723] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:13:36.467206 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1363/2724] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:36 2022] DELETE /identity/v3/users/3a0be5ca18164c849baa1760fb297479 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:36.468864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.470640 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.471275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.471565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.471877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.484339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5c40e-dffb-4bd4-9381-2c8544b2d37f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.548423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80be116-7109-4d99-8acc-4180a80914b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.549100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80be116-7109-4d99-8acc-4180a80914b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.549376 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80be116-7109-4d99-8acc-4180a80914b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.549651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80be116-7109-4d99-8acc-4180a80914b6 admin admin] PATH_INFO: `/v3/projects/b53959846549468593cd3a51af97e967/users/38facf5805b24d78a221f21d45ce5b98/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.566671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80be116-7109-4d99-8acc-4180a80914b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b53959846549468593cd3a51af97e967, user_id=38facf5805b24d78a221f21d45ce5b98, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.569367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80be116-7109-4d99-8acc-4180a80914b6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.586143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1362/2725] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:36 2022] PUT /identity/v3/projects/b53959846549468593cd3a51af97e967/users/38facf5805b24d78a221f21d45ce5b98/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:36.598934 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.599215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.599488 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.615596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:36.616806 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.629732 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] MFA Rules not processed for user `38facf5805b24d78a221f21d45ce5b98`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:36.630932 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.676266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:36.677764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1364/2726] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:36.688358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.689715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:36.695523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1363/2727] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:36.699888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.709897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c824f23-c1dc-49e4-9801-319a1d5bb5cc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.714397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.714985 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.715912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.716261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.717083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.719634 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.748962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a26cf-3ad3-41fe-a6b6-0cfef3529ad7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.751035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1364/2728] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:13:36.755094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.763199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:36.805964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.806566 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.806843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.807106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.823484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c84d7-1193-4333-85be-96a7867d6859 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.828561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.829162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.829443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.829719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.830350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.832285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.855368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1365/2729] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:36 2022] POST /identity/v3/users => generated 310 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:13:36.859857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.869532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8060cb-7277-448d-bcb8-c3126608147a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.871230 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.873718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.874266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.874900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.875313 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.875623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.876295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.877863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.884475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1365/2730] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:36.887921 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1366/2731] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:13:36.892028 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.901289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee27c2-7bc4-4156-ae89-88bd8d843522 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:36.906795 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:36.907446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:36.907856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:36.908239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] PATH_INFO: `/v3/projects/c35878b5ee504f2e8011c80d2bf223aa/users/4b699b4e2cf445c58c22f5650e3af425/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:36.924497 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35878b5ee504f2e8011c80d2bf223aa, user_id=4b699b4e2cf445c58c22f5650e3af425, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:36.927149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:36.941038 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1366/2732] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:36 2022] PUT /identity/v3/projects/c35878b5ee504f2e8011c80d2bf223aa/users/4b699b4e2cf445c58c22f5650e3af425/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:36.944876 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:36.957084 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df174811-d392-4d5b-afd7-ee99d8d437c7 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.019273 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e451bd-f825-4c57-8cac-6040fd9879ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.020073 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e451bd-f825-4c57-8cac-6040fd9879ab admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.020180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e451bd-f825-4c57-8cac-6040fd9879ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.020448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e451bd-f825-4c57-8cac-6040fd9879ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.021092 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e451bd-f825-4c57-8cac-6040fd9879ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.022688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e451bd-f825-4c57-8cac-6040fd9879ab admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.036804 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1367/2733] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:37.038157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.052217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67044f-c508-42b9-aca3-42bb36d3a22d admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.057615 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.057948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.058294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.058633 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] PATH_INFO: `/v3/projects/c35878b5ee504f2e8011c80d2bf223aa/users/4b699b4e2cf445c58c22f5650e3af425/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.060911 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c35878b5ee504f2e8011c80d2bf223aa, user_id=4b699b4e2cf445c58c22f5650e3af425, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.063600 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.074275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1367/2734] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:37 2022] PUT /identity/v3/projects/c35878b5ee504f2e8011c80d2bf223aa/users/4b699b4e2cf445c58c22f5650e3af425/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:37.086082 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.086536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.086820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.116910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] MFA Rules not processed for user `4b699b4e2cf445c58c22f5650e3af425`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:37.118198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.150880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:37.173185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.175068 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.198367 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1368/2735] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:37.210848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.212268 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.243083 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefc4ad-dfb9-4441-bc70-cbf0b5bf6f98 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.249310 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.249948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.250343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.250798 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] PATH_INFO: `/v3/projects/a1cc280f321b4896a9af2d59312f3fbe` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.252294 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cc280f321b4896a9af2d59312f3fbe)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.254269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.273008 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c782f-13bd-4da5-825f-303f968ec593 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.276812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a75276-ab90-42fc-8415-dcab9d3450f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.278394 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a75276-ab90-42fc-8415-dcab9d3450f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.278775 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a75276-ab90-42fc-8415-dcab9d3450f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.279057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a75276-ab90-42fc-8415-dcab9d3450f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.290928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a75276-ab90-42fc-8415-dcab9d3450f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.292419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a75276-ab90-42fc-8415-dcab9d3450f8 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.293516 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:37.303469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1369/2736] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:37 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:37.308428 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cc280f321b4896a9af2d59312f3fbe', 'request_id': 'req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:37.315910 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cc280f321b4896a9af2d59312f3fbe', 'request_id': 'req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:37.321962 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cc280f321b4896a9af2d59312f3fbe', 'request_id': 'req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:37.336627 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] The token cache is being invalidate because project a1cc280f321b4896a9af2d59312f3fbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:37.336914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:37.337575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ec60765f-5ca7-40b7-ba70-68c51d0a57c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:37.339111 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1368/2737] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:37 2022] DELETE /identity/v3/projects/a1cc280f321b4896a9af2d59312f3fbe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:37.344902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.345177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.345447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.366013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:37.367203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.428595 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1370/2738] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:37.436970 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.437167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.437446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.458470 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:37.459575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.462558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.463882 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.467742 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1369/2739] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:37.472064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.481314 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35eedc-f07d-4ed7-ba7d-883198b84a55 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.486342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.486928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.487216 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.487451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.488110 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:37.488411 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.490000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.502280 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1370/2740] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:13:37.506810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.517727 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90e6d2-3161-4ec8-a832-3748a67f6eef admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.522825 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.523002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab11f263-101b-4d15-8051-563940d5ba3f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.523421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.523701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.524022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.524653 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.526518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.526638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.527108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.527735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.528040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.529766 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.558108 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:37.565702 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1371/2741] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:37.567163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.602281 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.603762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.613645 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1371/2742] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:37 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:13:37.638557 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e826b1-d1f0-47bb-81a6-bf63634581d3 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.643518 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.644149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.644431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.644705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.645338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.647281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.670528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1372/2743] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:13:37.674940 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.687979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd994e1-f1f4-4480-b74e-017a0dd48977 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.692866 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.693467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.693750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.694025 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.694676 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.696277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.707017 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1372/2744] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:37.711029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.720955 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c349ba-e7d5-41fb-96be-7f895b126927 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.725582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683c8f1-1c14-4cbd-82c5-266f27d75f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.726182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683c8f1-1c14-4cbd-82c5-266f27d75f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.726455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683c8f1-1c14-4cbd-82c5-266f27d75f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.726720 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683c8f1-1c14-4cbd-82c5-266f27d75f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.727355 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683c8f1-1c14-4cbd-82c5-266f27d75f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.728952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683c8f1-1c14-4cbd-82c5-266f27d75f68 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.739440 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1373/2745] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:37.744669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:37.755079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c8692-1117-4181-a915-4bcd3245e3e1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:37.759950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:37.760627 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.760906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.761178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] PATH_INFO: `/v3/projects/2a29c768aa7340a2887b1c9d42618c4e/users/f1ddec8387114bd2bb48bf2dc3d10032/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.778971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a29c768aa7340a2887b1c9d42618c4e, user_id=f1ddec8387114bd2bb48bf2dc3d10032, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:37.782884 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:37.800437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1373/2746] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:37 2022] PUT /identity/v3/projects/2a29c768aa7340a2887b1c9d42618c4e/users/f1ddec8387114bd2bb48bf2dc3d10032/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:37.812757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:37.813065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:37.813333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:37.844669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] MFA Rules not processed for user `f1ddec8387114bd2bb48bf2dc3d10032`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:37.845934 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:37.906476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:37.935342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1374/2747] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:37.939873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.005348 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fab9bd-3686-43bd-a9a5-d2cca928b8bb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.010309 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.010951 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.011226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.011503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.012172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.014168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.040650 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1374/2748] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:37 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:38.045513 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.056072 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0ed9b9-c68d-4179-9b3f-3a04fcfced20 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.112755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.113363 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.113646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.113925 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.114624 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.116639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.138896 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1375/2749] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:38 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:38.143201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.154204 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b610ae-1c86-4cc9-aa7d-cc930ac83b22 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.159051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.159649 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.159929 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.160234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.160904 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.162468 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.174064 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1375/2750] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:38.178457 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.189093 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc9cf-857e-4004-946f-57f7ddf913eb admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.193660 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.194306 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.194584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.194856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] PATH_INFO: `/v3/projects/c79f3f4f095441d2baaa7226be9a712a/users/e106ede32fce49bbb0d781db93645810/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.209443 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79f3f4f095441d2baaa7226be9a712a, user_id=e106ede32fce49bbb0d781db93645810, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.212091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.225815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1376/2751] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:38 2022] PUT /identity/v3/projects/c79f3f4f095441d2baaa7226be9a712a/users/e106ede32fce49bbb0d781db93645810/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:38.229855 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.240500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3ddd0-05c4-4d32-8efe-55b80a64580d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.299176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6eeb00-8ca6-4dc6-ac67-66e74f56b21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.299779 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eeb00-8ca6-4dc6-ac67-66e74f56b21f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.300096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eeb00-8ca6-4dc6-ac67-66e74f56b21f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.300462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6eeb00-8ca6-4dc6-ac67-66e74f56b21f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.301152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eeb00-8ca6-4dc6-ac67-66e74f56b21f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.302774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6eeb00-8ca6-4dc6-ac67-66e74f56b21f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.308148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:38.314464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1376/2752] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:38.318769 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.329584 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a1699-7669-4515-b649-cd739d8ba323 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.334477 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.335144 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.335420 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.335685 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] PATH_INFO: `/v3/projects/c79f3f4f095441d2baaa7226be9a712a/users/e106ede32fce49bbb0d781db93645810/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.337651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c79f3f4f095441d2baaa7226be9a712a, user_id=e106ede32fce49bbb0d781db93645810, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.340796 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.351128 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1377/2753] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:38 2022] PUT /identity/v3/projects/c79f3f4f095441d2baaa7226be9a712a/users/e106ede32fce49bbb0d781db93645810/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:38.362045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.362330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.362630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.391447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] MFA Rules not processed for user `e106ede32fce49bbb0d781db93645810`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:38.392772 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:38.461433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1377/2754] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:38.473754 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.538158 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d533a4-9bab-43ff-95e6-784b2f0c6b9a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.542442 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.543035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.543351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.543618 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.552572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.553971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.564196 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1378/2755] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:38.639434 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:38.649015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f808c741-e664-440a-9644-a5e194c0cc44 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:38.652978 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf662d-1d63-4878-aac7-237b3eeff61a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:38.653642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf662d-1d63-4878-aac7-237b3eeff61a service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:38.653917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf662d-1d63-4878-aac7-237b3eeff61a service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:38.654193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf662d-1d63-4878-aac7-237b3eeff61a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:38.710063 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf662d-1d63-4878-aac7-237b3eeff61a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:38.711679 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf662d-1d63-4878-aac7-237b3eeff61a service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:38.720692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1378/2756] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:48.106968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:48.108502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:48.144125 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:48.175590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b0da7-1159-44b0-9a88-e1fb0d2706b3 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:48.180165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:48.180789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:48.181064 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:48.181357 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] PATH_INFO: `/v3/users/38facf5805b24d78a221f21d45ce5b98` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:48.182468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38facf5805b24d78a221f21d45ce5b98)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:48.184620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:48.232864 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38facf5805b24d78a221f21d45ce5b98'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.239901 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38facf5805b24d78a221f21d45ce5b98'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.247755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38facf5805b24d78a221f21d45ce5b98'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.260268 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38facf5805b24d78a221f21d45ce5b98'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.260989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38facf5805b24d78a221f21d45ce5b98'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.267412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1379/2757] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:48 2022] DELETE /identity/v3/users/38facf5805b24d78a221f21d45ce5b98 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:48.272216 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:48.272786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:48.272786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:48.293587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:48.294802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:48.357928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1379/2758] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:13:48.367617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:48.369114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:48.432941 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c11e7c-0234-4341-b1bd-8db886435795 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:48.436730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:48.437331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:48.437611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:48.437887 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:48.450489 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:48.451959 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:48.461865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1380/2759] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:48.750057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:48.751718 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:48.772295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6df0b-ccc9-4bc9-ad56-95aba852a6c9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:48.787010 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:48.787984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:48.788298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:48.788577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] PATH_INFO: `/v3/projects/b53959846549468593cd3a51af97e967` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:48.790576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b53959846549468593cd3a51af97e967)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:48.792717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:48.817697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:48.845439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b53959846549468593cd3a51af97e967', 'request_id': 'req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.853392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b53959846549468593cd3a51af97e967', 'request_id': 'req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.860304 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b53959846549468593cd3a51af97e967', 'request_id': 'req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.873040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] The token cache is being invalidate because project b53959846549468593cd3a51af97e967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:48.873448 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.874007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:48.875577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1380/2760] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:48 2022] DELETE /identity/v3/projects/b53959846549468593cd3a51af97e967 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:48.880190 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:48.892048 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac679e-5850-4980-8987-2351e9cedf63 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:48.947689 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:48.958534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:48.959233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:48.959621 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:48.960249 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] PATH_INFO: `/v3/users/4b699b4e2cf445c58c22f5650e3af425` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:48.962020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b699b4e2cf445c58c22f5650e3af425)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:48.964577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.016843 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b699b4e2cf445c58c22f5650e3af425'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.023914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b699b4e2cf445c58c22f5650e3af425'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.031487 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b699b4e2cf445c58c22f5650e3af425'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.043865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b699b4e2cf445c58c22f5650e3af425'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.044585 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-fca4c447-9f35-4caf-9cef-d9a2afdfaa14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b699b4e2cf445c58c22f5650e3af425'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.051305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1381/2761] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:48 2022] DELETE /identity/v3/users/4b699b4e2cf445c58c22f5650e3af425 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:49.330697 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.332456 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:49.349094 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4857c3-6ad3-4e8a-b31c-6c6aa3fbc55a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.419430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.420099 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.420390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.420664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] PATH_INFO: `/v3/projects/c35878b5ee504f2e8011c80d2bf223aa` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.421802 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] RBAC: Authorizing `identity:delete_project(project_id=c35878b5ee504f2e8011c80d2bf223aa)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.423711 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.470789 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c35878b5ee504f2e8011c80d2bf223aa', 'request_id': 'req-c098b9fd-8445-4afa-a91a-e3a4e2444fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.478556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c35878b5ee504f2e8011c80d2bf223aa', 'request_id': 'req-c098b9fd-8445-4afa-a91a-e3a4e2444fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.485390 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c35878b5ee504f2e8011c80d2bf223aa', 'request_id': 'req-c098b9fd-8445-4afa-a91a-e3a4e2444fda'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.496439 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] The token cache is being invalidate because project c35878b5ee504f2e8011c80d2bf223aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:49.496725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.497404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c098b9fd-8445-4afa-a91a-e3a4e2444fda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:49.498903 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1381/2762] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:49 2022] DELETE /identity/v3/projects/c35878b5ee504f2e8011c80d2bf223aa => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:49.503936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.504439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.504552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.524183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:49.525391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:49.585172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1382/2763] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:49.593579 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.593870 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.594149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.614482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:49.615692 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:49.624015 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1382/2764] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:49.628574 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.639382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dfb03-efcd-4ce9-95aa-8fb1d09d3622 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.644225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.644829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.645123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.645379 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.646026 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:49.646327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.647904 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.660670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1383/2765] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 11:13:49.666549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.678162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af2ac3-bfaa-48e0-909d-a700c4cd769f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.682747 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.683506 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.683979 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.684282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.684959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.686850 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.716422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1383/2766] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:49 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:49.720718 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.731563 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faba88e-1337-40ed-9981-df6b77cd49ff admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.795338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.795963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.796279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.796555 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.797215 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.799192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.809741 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:49.829213 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1384/2767] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:49 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:49.830580 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.846071 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c64cd6-efe8-46c7-86ad-8f273e618647 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.854852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.855330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.855613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.855890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.856591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.858208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.861341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:49.871168 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1384/2768] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:49.876220 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.888256 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab592-4444-4d18-95ca-df2a40872944 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.893426 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea32d1-73da-45ba-aa80-dc11b0624781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.894050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea32d1-73da-45ba-aa80-dc11b0624781 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.894396 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea32d1-73da-45ba-aa80-dc11b0624781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.894590 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea32d1-73da-45ba-aa80-dc11b0624781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.895292 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea32d1-73da-45ba-aa80-dc11b0624781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.897108 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea32d1-73da-45ba-aa80-dc11b0624781 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.910636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1385/2769] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:13:49.915705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:49.926926 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd421d1-0bb6-4b2e-8f4d-52f088796d52 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:49.932501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:49.933255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.933875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.934223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] PATH_INFO: `/v3/projects/cd0ccf4425cc4b5fad55ea8b854adb3c/users/7df5a90d2bef4ae0ad450abf248debb4/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:49.951091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd0ccf4425cc4b5fad55ea8b854adb3c, user_id=7df5a90d2bef4ae0ad450abf248debb4, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:49.953809 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:49.973208 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1385/2770] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:49 2022] PUT /identity/v3/projects/cd0ccf4425cc4b5fad55ea8b854adb3c/users/7df5a90d2bef4ae0ad450abf248debb4/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:49.985182 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:49.985462 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:49.985734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.018670 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] MFA Rules not processed for user `7df5a90d2bef4ae0ad450abf248debb4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:50.020057 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:50.100701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1386/2771] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:50.112459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:50.114262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:50.189741 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2386c7c7-8fa1-43c9-872b-329b95a6c848 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:50.193322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d34f6d-ac16-4f17-8e1e-cbe844ab8133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:50.193998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34f6d-ac16-4f17-8e1e-cbe844ab8133 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:50.194276 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34f6d-ac16-4f17-8e1e-cbe844ab8133 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:50.194546 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d34f6d-ac16-4f17-8e1e-cbe844ab8133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.204011 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34f6d-ac16-4f17-8e1e-cbe844ab8133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:50.205466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d34f6d-ac16-4f17-8e1e-cbe844ab8133 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:50.215041 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1386/2772] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:50.236540 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:50.301786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5767f-82f1-4d28-864c-97c3fa20cf3b None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:50.306368 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:50.306997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:50.307275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:50.307550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] PATH_INFO: `/v3/users/7df5a90d2bef4ae0ad450abf248debb4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.309262 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7df5a90d2bef4ae0ad450abf248debb4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:50.311159 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:50.358823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7df5a90d2bef4ae0ad450abf248debb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.365625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7df5a90d2bef4ae0ad450abf248debb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.373337 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7df5a90d2bef4ae0ad450abf248debb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.381065 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:50.388250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7df5a90d2bef4ae0ad450abf248debb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.389708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7df5a90d2bef4ae0ad450abf248debb4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.396546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1387/2773] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:50 2022] DELETE /identity/v3/users/7df5a90d2bef4ae0ad450abf248debb4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:50.401054 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:50.401915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:50.401915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.422302 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:50.423443 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:50.482196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1387/2774] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:50.492251 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:50.493799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:50.551320 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b6129-6e5d-44ae-b2d2-b7c7c584c770 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:50.555257 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d973ad-b54a-4776-8564-5a48aec0c80f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:50.555856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d973ad-b54a-4776-8564-5a48aec0c80f service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:50.556168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d973ad-b54a-4776-8564-5a48aec0c80f service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:50.556444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d973ad-b54a-4776-8564-5a48aec0c80f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.565732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d973ad-b54a-4776-8564-5a48aec0c80f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:50.567146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d973ad-b54a-4776-8564-5a48aec0c80f service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:50.577204 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1388/2775] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:50.848687 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:50.850171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:50.862626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503afacb-a2ea-4404-abfb-ab94e362b810 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:50.867821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:50.869464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:50.869755 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:50.870031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] PATH_INFO: `/v3/projects/cd0ccf4425cc4b5fad55ea8b854adb3c` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.871507 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd0ccf4425cc4b5fad55ea8b854adb3c)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:50.873578 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:50.922389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0ccf4425cc4b5fad55ea8b854adb3c', 'request_id': 'req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.929290 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0ccf4425cc4b5fad55ea8b854adb3c', 'request_id': 'req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.934111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:50.936686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0ccf4425cc4b5fad55ea8b854adb3c', 'request_id': 'req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.949545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] The token cache is being invalidate because project cd0ccf4425cc4b5fad55ea8b854adb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:50.950043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.950715 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-71fe0bfa-a568-4ed2-843c-fbebbe6b5869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:50.952244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1388/2776] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:50 2022] DELETE /identity/v3/projects/cd0ccf4425cc4b5fad55ea8b854adb3c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:50.957234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:50.957641 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:50.958024 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:50.977122 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:50.978304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:51.038453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1389/2777] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:51.046072 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.046357 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.046642 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.068663 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:51.069868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:51.079212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1389/2778] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:51.084919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.094750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1b08d8-a58b-4fdf-88f2-7d48227083ee None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.099397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.100050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.100334 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.100605 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.101254 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:51.101696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.103243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.116892 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1390/2779] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:51.122040 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.132229 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed7b69c-f2f0-4e07-8887-799e698636d8 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.138386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.139165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.139480 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.139752 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.140466 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.142366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.171253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1390/2780] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:51 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:51.175397 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.186300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72539bd3-3b2d-4afb-8a3e-7e9330efd121 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.250730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.251336 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.251611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.251880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.252548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.254437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.279898 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1391/2781] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:51 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:51.284267 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.295899 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce5737-92b8-475f-a686-7df16d7e802c admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.300778 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.301392 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.301656 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.301923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.302623 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.304286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.315461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1391/2782] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:51.319722 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.331352 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef2cbe-c8ae-42c7-9b14-5152ae678858 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.336738 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4859f8-30d9-4853-a6a7-eca9fa60974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.337350 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4859f8-30d9-4853-a6a7-eca9fa60974a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.337678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4859f8-30d9-4853-a6a7-eca9fa60974a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.338052 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4859f8-30d9-4853-a6a7-eca9fa60974a admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.338816 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4859f8-30d9-4853-a6a7-eca9fa60974a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.341059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4859f8-30d9-4853-a6a7-eca9fa60974a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.343541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:51.353529 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1392/2783] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:51.357807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.368565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df42c4d-ad6e-4488-9e7d-6b4c16878621 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.373615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.374359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.374705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.375048 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] PATH_INFO: `/v3/projects/0cae724f4cbb4be7b865635ef8f19bb7/users/bc02cb968a7d442b9bfbd0490011360e/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.391246 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cae724f4cbb4be7b865635ef8f19bb7, user_id=bc02cb968a7d442b9bfbd0490011360e, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.402622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.410046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1392/2784] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:51 2022] PUT /identity/v3/projects/0cae724f4cbb4be7b865635ef8f19bb7/users/bc02cb968a7d442b9bfbd0490011360e/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:51.421343 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.421772 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.422054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.451370 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] MFA Rules not processed for user `bc02cb968a7d442b9bfbd0490011360e`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:51.452639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:51.518505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1393/2785] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:51.531149 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:51.532883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:51.600673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13810475-4cb9-4539-b611-60e7e5a3ae1b admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:51.604562 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1f308-b88b-4053-9994-2cdc3d13f4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:51.605162 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1f308-b88b-4053-9994-2cdc3d13f4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:51.605442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1f308-b88b-4053-9994-2cdc3d13f4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:51.605788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1f308-b88b-4053-9994-2cdc3d13f4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:51.620999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1f308-b88b-4053-9994-2cdc3d13f4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:51.622528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c1f308-b88b-4053-9994-2cdc3d13f4e8 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:51.631944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1393/2786] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:54.274167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:54.342815 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917d7a1-e0ee-49a6-9047-0933c4c01d86 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:54.347422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:54.348153 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:54.348436 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:54.348709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] PATH_INFO: `/v3/users/bc02cb968a7d442b9bfbd0490011360e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:54.349821 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc02cb968a7d442b9bfbd0490011360e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:54.351750 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:54.397217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc02cb968a7d442b9bfbd0490011360e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.404940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc02cb968a7d442b9bfbd0490011360e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.413948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc02cb968a7d442b9bfbd0490011360e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.427468 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc02cb968a7d442b9bfbd0490011360e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.428187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc02cb968a7d442b9bfbd0490011360e'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.434662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1394/2787] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:13:54 2022] DELETE /identity/v3/users/bc02cb968a7d442b9bfbd0490011360e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:54.439198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:54.439727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:54.439727 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:54.459411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:54.460659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:54.518427 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1394/2788] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:54.528949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:54.530358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:54.540747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:54.591000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea4e1d2-5621-4a69-ba1d-25bb5022768f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:54.594696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374d34a-47fc-4df3-9c9c-3936541aaaed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:54.595291 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d34a-47fc-4df3-9c9c-3936541aaaed service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:54.595572 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d34a-47fc-4df3-9c9c-3936541aaaed service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:54.595846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374d34a-47fc-4df3-9c9c-3936541aaaed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:54.604719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d34a-47fc-4df3-9c9c-3936541aaaed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:54.606090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374d34a-47fc-4df3-9c9c-3936541aaaed service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:54.615252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1395/2789] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:13:54.889824 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:54.891528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:54.904543 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41d5fb-2182-4d6f-b36e-4d75d9ae27c5 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:54.909447 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:54.910187 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:54.910472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:54.910750 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] PATH_INFO: `/v3/projects/0cae724f4cbb4be7b865635ef8f19bb7` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:54.911896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cae724f4cbb4be7b865635ef8f19bb7)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:54.913825 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:54.938976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:54.962181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cae724f4cbb4be7b865635ef8f19bb7', 'request_id': 'req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.969565 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cae724f4cbb4be7b865635ef8f19bb7', 'request_id': 'req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.974994 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cae724f4cbb4be7b865635ef8f19bb7', 'request_id': 'req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.990763 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] The token cache is being invalidate because project 0cae724f4cbb4be7b865635ef8f19bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:13:54.991058 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.991706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1ca4403a-dd2f-466e-9d05-22cb766a4aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:13:54.993198 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1395/2790] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:13:54 2022] DELETE /identity/v3/projects/0cae724f4cbb4be7b865635ef8f19bb7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:13:54.998500 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:54.998782 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:54.999053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.019283 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:55.020490 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:55.080114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1396/2791] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:55.087821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.088143 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.088421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.110501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:55.111701 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:55.121075 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1396/2792] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:55.125865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.137474 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa9387-ef30-42eb-8f25-b9b891ab62af None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.142136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.142728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.143003 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.143278 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.144053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:13:55.144356 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.145917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.158940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1397/2793] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:13:55.163283 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.173504 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa65d7e-a37b-4507-b8bb-6ed26a72d38f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.178128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.178720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.178997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.179266 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.179892 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.181784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.210776 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1397/2794] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:13:55 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:13:55.215071 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.226010 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853273e-14c3-4665-a769-5b0670f32f7a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.284997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.285678 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.285867 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.286136 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.286793 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.288728 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.312603 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1398/2795] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:13:55 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:13:55.316971 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.328336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bb54f-df68-4616-97a6-db350f5d96da admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.333295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.333896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.334178 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.334453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.335130 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.336786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.347613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1398/2796] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:55.352142 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.364733 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f0c874-e90c-473f-8d02-0fbf8ccf879a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.369622 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7bd204-52bd-4a5b-803a-facaff523654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.370264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bd204-52bd-4a5b-803a-facaff523654 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.371226 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bd204-52bd-4a5b-803a-facaff523654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.371437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7bd204-52bd-4a5b-803a-facaff523654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.372124 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bd204-52bd-4a5b-803a-facaff523654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.373725 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7bd204-52bd-4a5b-803a-facaff523654 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.386534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1399/2797] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:13:55.390534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.400532 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62264e8-035f-475f-afd8-65e1682a781f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.406068 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.406730 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.407013 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.407287 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] PATH_INFO: `/v3/projects/03198546794f42a0a28fed91f6eb523a/users/c4a5df3b1e19450093d7d4d1af03843a/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.425559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03198546794f42a0a28fed91f6eb523a, user_id=c4a5df3b1e19450093d7d4d1af03843a, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.428404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.447990 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1399/2798] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:13:55 2022] PUT /identity/v3/projects/03198546794f42a0a28fed91f6eb523a/users/c4a5df3b1e19450093d7d4d1af03843a/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:13:55.457079 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555e356-2bcd-4055-a33a-71a1025ae51f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.457928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555e356-2bcd-4055-a33a-71a1025ae51f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.457928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1555e356-2bcd-4055-a33a-71a1025ae51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.491331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1555e356-2bcd-4055-a33a-71a1025ae51f None None] MFA Rules not processed for user `c4a5df3b1e19450093d7d4d1af03843a`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:13:55.492708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1555e356-2bcd-4055-a33a-71a1025ae51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:55.497414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:13:55.561721 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1400/2799] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:13:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:13:55.572550 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:13:55.574363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:13:55.630315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f0627-373d-42fc-b470-59d8b7e95ed0 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:13:55.633898 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:13:55.634496 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:13:55.634777 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:13:55.635053 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:13:55.643669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:13:55.645126 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:13:55.654661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1400/2800] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:13:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:05.559106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=25 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:05.649867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:15.527390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde72fe-050a-4d61-8fcf-5f17fe6f4483 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:15.527923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde72fe-050a-4d61-8fcf-5f17fe6f4483 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:15.527923 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde72fe-050a-4d61-8fcf-5f17fe6f4483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:15.554996 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-ebde72fe-050a-4d61-8fcf-5f17fe6f4483 None None] MFA Rules not processed for user `16696c34de254f3babf538c879f2ce49`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:15.556238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-ebde72fe-050a-4d61-8fcf-5f17fe6f4483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:15.628351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1401/2801] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:16.048098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:16.111620 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176640ba-801d-47c8-b131-66c4f1dfe488 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:16.116308 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:16.116935 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:16.117212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:16.117486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] PATH_INFO: `/v3/users/c4a5df3b1e19450093d7d4d1af03843a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:16.118604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4a5df3b1e19450093d7d4d1af03843a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:16.121384 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:16.173284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4a5df3b1e19450093d7d4d1af03843a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.173952 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4a5df3b1e19450093d7d4d1af03843a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.182386 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4a5df3b1e19450093d7d4d1af03843a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.196671 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4a5df3b1e19450093d7d4d1af03843a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.203993 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4a5df3b1e19450093d7d4d1af03843a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.210245 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1401/2802] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:16 2022] DELETE /identity/v3/users/c4a5df3b1e19450093d7d4d1af03843a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:16.215058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:16.215342 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:16.215617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:16.235862 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:16.237564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:16.305514 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1402/2803] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:14:16.315042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:16.316693 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:16.377900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c6591-1648-433e-9d45-43299db1e6bb admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:16.381617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:16.382233 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:16.382500 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:16.382774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:16.393706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:16.395323 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:16.408086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1402/2804] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:16.672556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:16.673986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:16.686625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e3f4b-ef5a-4272-b391-8ff3becff041 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:16.691394 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:16.692115 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:16.692467 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:16.692810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] PATH_INFO: `/v3/projects/03198546794f42a0a28fed91f6eb523a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:16.694011 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03198546794f42a0a28fed91f6eb523a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:16.695964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:16.746054 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03198546794f42a0a28fed91f6eb523a', 'request_id': 'req-e3bd4507-a267-4c90-8047-f3a6eb7c3730'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.752441 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03198546794f42a0a28fed91f6eb523a', 'request_id': 'req-e3bd4507-a267-4c90-8047-f3a6eb7c3730'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.758414 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03198546794f42a0a28fed91f6eb523a', 'request_id': 'req-e3bd4507-a267-4c90-8047-f3a6eb7c3730'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.771106 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] The token cache is being invalidate because project 03198546794f42a0a28fed91f6eb523a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:16.771400 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.772031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e3bd4507-a267-4c90-8047-f3a6eb7c3730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:16.773945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1403/2805] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:16 2022] DELETE /identity/v3/projects/03198546794f42a0a28fed91f6eb523a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:24.745381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:24.747119 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:24.760147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170ff7a-702d-4a05-b658-ebc444f5f81f service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:24.780795 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:24.826837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:24.827469 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:24.827753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:24.828052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] PATH_INFO: `/v3/users/f1ddec8387114bd2bb48bf2dc3d10032` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:24.829213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ddec8387114bd2bb48bf2dc3d10032)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:24.831135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:24.880368 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ddec8387114bd2bb48bf2dc3d10032'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:24.880947 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ddec8387114bd2bb48bf2dc3d10032'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:24.888995 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ddec8387114bd2bb48bf2dc3d10032'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:24.903254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ddec8387114bd2bb48bf2dc3d10032'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:24.910739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ddec8387114bd2bb48bf2dc3d10032'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:24.916784 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1403/2806] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:24 2022] DELETE /identity/v3/users/f1ddec8387114bd2bb48bf2dc3d10032 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:24.921476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634f894-380d-4c30-b4ec-07574477134e None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:24.921974 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634f894-380d-4c30-b4ec-07574477134e None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:24.922076 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634f894-380d-4c30-b4ec-07574477134e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:24.941250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-c634f894-380d-4c30-b4ec-07574477134e None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:24.942454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c634f894-380d-4c30-b4ec-07574477134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:24.968431 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:25.006285 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1404/2807] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:25.016788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:25.018226 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:25.078811 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79195120-a5f2-46b0-a67c-404ca6ede694 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:25.082980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:25.083587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:25.083869 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:25.084176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:25.094141 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:25.095582 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:25.105739 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1404/2808] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:25.383155 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634f894-380d-4c30-b4ec-07574477134e None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:25.384712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-c634f894-380d-4c30-b4ec-07574477134e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:25.399030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634f894-380d-4c30-b4ec-07574477134e None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:25.404671 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:25.406264 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:25.406439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:25.407192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] PATH_INFO: `/v3/projects/2a29c768aa7340a2887b1c9d42618c4e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:25.408386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a29c768aa7340a2887b1c9d42618c4e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:25.410389 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:25.460121 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a29c768aa7340a2887b1c9d42618c4e', 'request_id': 'req-f39b8662-a245-461e-85f2-59efbf88c2ac'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.472693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a29c768aa7340a2887b1c9d42618c4e', 'request_id': 'req-f39b8662-a245-461e-85f2-59efbf88c2ac'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.482171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a29c768aa7340a2887b1c9d42618c4e', 'request_id': 'req-f39b8662-a245-461e-85f2-59efbf88c2ac'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.497223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] The token cache is being invalidate because project 2a29c768aa7340a2887b1c9d42618c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:25.497508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.498137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.499645 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1405/2809] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:25 2022] DELETE /identity/v3/projects/2a29c768aa7340a2887b1c9d42618c4e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:25.503652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:25.514200 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8633c-e780-4923-9861-4be97ebbd9e0 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:25.561028 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:25.578036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:25.578667 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:25.578950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:25.579223 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] PATH_INFO: `/v3/users/e106ede32fce49bbb0d781db93645810` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:25.580383 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e106ede32fce49bbb0d781db93645810)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:25.582303 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:25.629812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e106ede32fce49bbb0d781db93645810'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.630464 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e106ede32fce49bbb0d781db93645810'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.638485 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e106ede32fce49bbb0d781db93645810'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.651714 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e106ede32fce49bbb0d781db93645810'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.658612 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-1d739c48-b26e-45b5-bafa-0184347b707e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e106ede32fce49bbb0d781db93645810'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:25.664888 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1405/2810] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:25 2022] DELETE /identity/v3/users/e106ede32fce49bbb0d781db93645810 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:26.271224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.272726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:26.290168 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b8662-a245-461e-85f2-59efbf88c2ac admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.352690 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.353243 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.353530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.353807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] PATH_INFO: `/v3/projects/c79f3f4f095441d2baaa7226be9a712a` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.355114 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c79f3f4f095441d2baaa7226be9a712a)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.357066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.405001 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c79f3f4f095441d2baaa7226be9a712a', 'request_id': 'req-ee7e857f-91f3-4ebd-b051-96b51ffdb690'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:26.411681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c79f3f4f095441d2baaa7226be9a712a', 'request_id': 'req-ee7e857f-91f3-4ebd-b051-96b51ffdb690'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:26.413066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:26.418546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c79f3f4f095441d2baaa7226be9a712a', 'request_id': 'req-ee7e857f-91f3-4ebd-b051-96b51ffdb690'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:26.430879 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] The token cache is being invalidate because project c79f3f4f095441d2baaa7226be9a712a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:26.431166 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:26.431807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-ee7e857f-91f3-4ebd-b051-96b51ffdb690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:26.433575 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1406/2811] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:26 2022] DELETE /identity/v3/projects/c79f3f4f095441d2baaa7226be9a712a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:14:26.438419 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.438902 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.439002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.459472 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:26.460700 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:26.519219 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1406/2812] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:26.527920 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.528232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.528507 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.548034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:26.549239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:26.558030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1407/2813] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:26.562407 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.572274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43585df-8e6c-4382-b813-778cd1be65a9 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.577518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.578114 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.578396 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.578673 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.579331 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:14:26.579638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.581289 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.593672 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1407/2814] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:14:26.598223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.608279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e9ff8-caea-4eca-9796-9432b5554158 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.613167 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.613763 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.614044 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.614317 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.614975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.616906 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.645536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1408/2815] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:26 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:26.650708 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.662176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4506f5-1ca4-4d7d-ac16-46dd7785e1d1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.712180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:26.728153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.729831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.730111 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.730375 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.731117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.733117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.755593 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1408/2816] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:26 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:26.760029 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.770510 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c6c5-724a-49b2-800b-d76fa18f06a2 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.775519 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.776730 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.777012 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.777287 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.777943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.779552 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.790212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1409/2817] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:26.795234 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.805421 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a71080-12f1-4fb7-8234-17e1fa1f0b0d admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.811788 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d830f963-fdfb-42e1-85cd-ded87f017b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.812761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830f963-fdfb-42e1-85cd-ded87f017b23 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.813042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830f963-fdfb-42e1-85cd-ded87f017b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.813315 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d830f963-fdfb-42e1-85cd-ded87f017b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.813977 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830f963-fdfb-42e1-85cd-ded87f017b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.817172 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d830f963-fdfb-42e1-85cd-ded87f017b23 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.831411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1409/2818] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:26.835840 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:26.846532 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23986056-6419-4f03-b17b-5ea5d71a9b47 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:26.851212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:26.851852 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.852160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.852435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] PATH_INFO: `/v3/projects/3d6e4465004e46b29bf3f132b64cb460/users/aea3501089544eff89f17157e00bf757/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.868245 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6e4465004e46b29bf3f132b64cb460, user_id=aea3501089544eff89f17157e00bf757, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:26.870914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:26.885913 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1410/2819] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:26 2022] PUT /identity/v3/projects/3d6e4465004e46b29bf3f132b64cb460/users/aea3501089544eff89f17157e00bf757/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:26.896381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:26.896668 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:26.896946 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:26.925896 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] MFA Rules not processed for user `aea3501089544eff89f17157e00bf757`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:26.927096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:26.993524 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1410/2820] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:26.998279 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.059829 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e022f5-b29b-4190-b037-e5a1a3621857 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.064954 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.065569 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.065877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.066150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.066802 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.068776 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.097856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1411/2821] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:26 2022] POST /identity/v3/projects => generated 393 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:27.102242 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.114425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457dbd5b-9a8c-4e70-b6f1-3b56150a402a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.178131 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.178743 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.179026 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.179296 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.179943 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.181872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.205508 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1411/2822] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:27 2022] POST /identity/v3/users => generated 303 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:27.210241 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.223598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654300f-ab19-4391-a99d-a44d0c7d3a32 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.228611 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.229326 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.229700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.229975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.230651 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.232272 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.243482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1412/2823] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:27.248298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.258154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f9352-0cfe-4c8c-88af-403ae379c586 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.263639 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.264339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.264638 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.264910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] PATH_INFO: `/v3/projects/f843b98a7cbb4d7fae293341d5f8a93e/users/9ad6501034e64af789d707b0749ba31a/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.282991 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f843b98a7cbb4d7fae293341d5f8a93e, user_id=9ad6501034e64af789d707b0749ba31a, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.286288 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.302880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1412/2824] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:27 2022] PUT /identity/v3/projects/f843b98a7cbb4d7fae293341d5f8a93e/users/9ad6501034e64af789d707b0749ba31a/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:27.308331 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.318698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21428b57-f3d3-4497-883f-a111fe70bdd4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.370891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:27.382746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c95c95-1a17-441b-a434-a9eab4cebe4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.383380 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c95c95-1a17-441b-a434-a9eab4cebe4f admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.383666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c95c95-1a17-441b-a434-a9eab4cebe4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.383943 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c95c95-1a17-441b-a434-a9eab4cebe4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.384650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c95c95-1a17-441b-a434-a9eab4cebe4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.387382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c95c95-1a17-441b-a434-a9eab4cebe4f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.398699 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1413/2825] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:27.405436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.417254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5dca04-0fae-4c35-b5eb-61799966f7c6 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.422252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.422928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.423205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.423481 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] PATH_INFO: `/v3/projects/f843b98a7cbb4d7fae293341d5f8a93e/users/9ad6501034e64af789d707b0749ba31a/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.425458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f843b98a7cbb4d7fae293341d5f8a93e, user_id=9ad6501034e64af789d707b0749ba31a, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.428101 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.439388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1413/2826] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:27 2022] PUT /identity/v3/projects/f843b98a7cbb4d7fae293341d5f8a93e/users/9ad6501034e64af789d707b0749ba31a/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:27.451847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.452175 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.452451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.482747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] MFA Rules not processed for user `9ad6501034e64af789d707b0749ba31a`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:27.484053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:27.552502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1414/2827] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:27.564423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.566400 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:27.582212 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:27.631534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ac435-ba9e-4d44-9234-10971ea988c9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.635091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.636045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.636336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.636615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.645872 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.647299 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.658186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1414/2828] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:27 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:27.705131 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:27.706550 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:27.719319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da154e-f006-4624-a46c-7a64ce73460d None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:27.722982 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223692d-d610-4cb0-adff-d576f43f7837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:27.723578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223692d-d610-4cb0-adff-d576f43f7837 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:27.723857 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223692d-d610-4cb0-adff-d576f43f7837 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:27.724164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223692d-d610-4cb0-adff-d576f43f7837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:27.787789 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223692d-d610-4cb0-adff-d576f43f7837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:27.789354 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223692d-d610-4cb0-adff-d576f43f7837 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:27.800527 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1415/2829] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:27 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:31.176606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:31.250437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f032056-b57b-4722-98f1-06c772087d78 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:31.254851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:31.255478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:31.255761 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:31.256057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] PATH_INFO: `/v3/users/aea3501089544eff89f17157e00bf757` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:31.257192 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aea3501089544eff89f17157e00bf757)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:31.259128 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:31.310684 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aea3501089544eff89f17157e00bf757'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:31.311604 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aea3501089544eff89f17157e00bf757'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:31.320339 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aea3501089544eff89f17157e00bf757'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:31.335478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aea3501089544eff89f17157e00bf757'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:31.349213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aea3501089544eff89f17157e00bf757'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:31.356931 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1415/2830] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:31 2022] DELETE /identity/v3/users/aea3501089544eff89f17157e00bf757 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:31.361927 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:31.362799 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:31.362987 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:31.384629 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:31.385973 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:31.456907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1416/2831] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:31.464670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:31.467586 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:31.520615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:31.547917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce315-2f42-4dc0-b723-34db5b82d276 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:31.551643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:31.552274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:31.552556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:31.552848 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:31.563249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:31.564803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:31.575608 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1416/2832] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:32.012333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:32.013596 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:32.025775 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2f481-5749-44be-8687-7b27a7171340 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:32.034861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:32.034861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:32.034861 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:32.035312 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] PATH_INFO: `/v3/projects/3d6e4465004e46b29bf3f132b64cb460` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:32.036461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6e4465004e46b29bf3f132b64cb460)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:32.038533 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:32.048761 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=100 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:32.095289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6e4465004e46b29bf3f132b64cb460', 'request_id': 'req-e27c27c6-caf4-446a-939e-92a95bf38027'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.102587 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6e4465004e46b29bf3f132b64cb460', 'request_id': 'req-e27c27c6-caf4-446a-939e-92a95bf38027'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.109485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6e4465004e46b29bf3f132b64cb460', 'request_id': 'req-e27c27c6-caf4-446a-939e-92a95bf38027'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.123089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] The token cache is being invalidate because project 3d6e4465004e46b29bf3f132b64cb460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:32.123385 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.124200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.125784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1417/2833] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:32 2022] DELETE /identity/v3/projects/3d6e4465004e46b29bf3f132b64cb460 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:32.130023 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:32.142425 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9526d3-ce49-4635-b484-8ec938cbfbb5 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:32.204527 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:32.205295 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:32.205422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:32.205689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] PATH_INFO: `/v3/users/9ad6501034e64af789d707b0749ba31a` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:32.206880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad6501034e64af789d707b0749ba31a)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:32.208851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:32.258240 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad6501034e64af789d707b0749ba31a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.258916 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad6501034e64af789d707b0749ba31a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.267077 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad6501034e64af789d707b0749ba31a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.282918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad6501034e64af789d707b0749ba31a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.290528 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-beeef9f9-9b67-4a6a-8d1a-2e3df0815b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad6501034e64af789d707b0749ba31a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.297142 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1417/2834] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:32 2022] DELETE /identity/v3/users/9ad6501034e64af789d707b0749ba31a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:32.593363 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:32.594807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:32.608353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c27c6-caf4-446a-939e-92a95bf38027 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:32.672176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:32.672839 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:32.673145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:32.673427 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] PATH_INFO: `/v3/projects/f843b98a7cbb4d7fae293341d5f8a93e` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:32.674636 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f843b98a7cbb4d7fae293341d5f8a93e)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:32.676588 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:32.737850 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f843b98a7cbb4d7fae293341d5f8a93e', 'request_id': 'req-904ace08-e13d-4295-a507-2d46c10551c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.746564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f843b98a7cbb4d7fae293341d5f8a93e', 'request_id': 'req-904ace08-e13d-4295-a507-2d46c10551c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.754508 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f843b98a7cbb4d7fae293341d5f8a93e', 'request_id': 'req-904ace08-e13d-4295-a507-2d46c10551c4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.771952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] The token cache is being invalidate because project f843b98a7cbb4d7fae293341d5f8a93e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:32.772445 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.773299 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-904ace08-e13d-4295-a507-2d46c10551c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:32.775676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1418/2835] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:32 2022] DELETE /identity/v3/projects/f843b98a7cbb4d7fae293341d5f8a93e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:32.783286 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:32.784038 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:32.784669 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:32.819043 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:32.821098 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:32.899773 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:32.914678 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1418/2836] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:32.923639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:32.923971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:32.924221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:32.946015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:32.947239 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:32.955297 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1419/2837] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:32.960486 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:32.970858 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6227e-03f8-40d6-8c8a-fa46f33b622f None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:32.977688 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:32.978689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:32.980238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:32.980702 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:32.981799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81463) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:14:32.982314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:32.985293 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.005933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1419/2838] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:14:33.011908 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.032787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2161c-8977-4ac4-88eb-022e23bfd762 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.040916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.043449 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.043928 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.044410 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.047058 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.051140 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.076386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:33.089305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1420/2839] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:33 2022] POST /identity/v3/projects => generated 387 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:33.094103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.111301 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2cb37-26e7-425f-a95e-dd9497b78aa4 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.185334 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.186154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.186560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.186953 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.187883 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.190056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.226248 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1420/2840] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:33 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:33.231193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.243091 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb220f7-036e-4605-a484-c8b4d7a0511f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.248724 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.249340 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.250399 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.250676 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.251328 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.252980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.265640 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1421/2841] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:33.270852 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.285686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665490dc-1d09-4b51-8373-4e44497bee4f admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.292166 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71d06e-7b14-4e90-bade-fd90d94585e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.292875 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71d06e-7b14-4e90-bade-fd90d94585e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.293148 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71d06e-7b14-4e90-bade-fd90d94585e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.293424 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71d06e-7b14-4e90-bade-fd90d94585e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.294083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71d06e-7b14-4e90-bade-fd90d94585e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.295717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71d06e-7b14-4e90-bade-fd90d94585e3 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.308193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1421/2842] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:33.312329 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.324865 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1236cda-bc66-4816-9bb1-74341de56c4e admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.330406 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.331250 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.331648 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.332031 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] PATH_INFO: `/v3/projects/664876d2894c48ea8745c0b89ce2d23d/users/c77f7154c18c4151b39b9a63e90f1189/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.351362 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] RBAC: Authorizing `identity:create_grant(project_id=664876d2894c48ea8745c0b89ce2d23d, user_id=c77f7154c18c4151b39b9a63e90f1189, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.354846 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.373968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1422/2843] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:33 2022] PUT /identity/v3/projects/664876d2894c48ea8745c0b89ce2d23d/users/c77f7154c18c4151b39b9a63e90f1189/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:33.394471 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.394949 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.395437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.434840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] MFA Rules not processed for user `c77f7154c18c4151b39b9a63e90f1189`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:33.436135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:33.519873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1422/2844] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:33.524118 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.632120 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449df53-958f-4e6d-a8cc-0af93494e15e admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.637597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.638232 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.638687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.638963 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.639617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.641548 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.672234 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1423/2845] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:33 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:14:33.677175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.692249 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9867cc2-c873-4a5e-a341-6b9463c42008 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.752997 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:33.766536 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.767282 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.767641 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.767980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] PATH_INFO: `/v3/users` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.768698 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.770644 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.808918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1423/2846] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:33 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:33.814900 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.827150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d2b4d-a777-4bf0-9969-d813fccd8ed3 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.832475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.833145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.833553 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.833980 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.834748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.836655 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.851849 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1424/2847] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:33.857320 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.876109 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3bd29-60b5-425f-8e5a-1d9616b50052 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:33.884100 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:33.885261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:33.885748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:33.886241 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] PATH_INFO: `/v3/projects/231ba747c0474f6198d937c293e196f4/users/e938e615637f4c9291c60df5dfeb06ea/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:33.912344 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=231ba747c0474f6198d937c293e196f4, user_id=e938e615637f4c9291c60df5dfeb06ea, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:33.915341 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:33.939261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1424/2848] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:33 2022] PUT /identity/v3/projects/231ba747c0474f6198d937c293e196f4/users/e938e615637f4c9291c60df5dfeb06ea/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:33.945163 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:33.958221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f22ed9d-cdc9-4da2-9a49-e340fb36d006 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:34.077885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7944f-94aa-4ca0-9714-65bc74fc5559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:34.078895 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7944f-94aa-4ca0-9714-65bc74fc5559 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:34.079371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7944f-94aa-4ca0-9714-65bc74fc5559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:34.079871 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7944f-94aa-4ca0-9714-65bc74fc5559 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:34.081119 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7944f-94aa-4ca0-9714-65bc74fc5559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:34.084171 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7944f-94aa-4ca0-9714-65bc74fc5559 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:34.099069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1425/2849] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:34.105462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:34.123810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b1092-82b9-441a-9fee-a598f541c51a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:34.132769 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:34.133998 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:34.134474 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:34.134950 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] PATH_INFO: `/v3/projects/231ba747c0474f6198d937c293e196f4/users/e938e615637f4c9291c60df5dfeb06ea/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:34.138704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=231ba747c0474f6198d937c293e196f4, user_id=e938e615637f4c9291c60df5dfeb06ea, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:34.142121 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:34.159413 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1425/2850] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:34 2022] PUT /identity/v3/projects/231ba747c0474f6198d937c293e196f4/users/e938e615637f4c9291c60df5dfeb06ea/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:34.171240 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:34.171639 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:34.171801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:34.205333 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] MFA Rules not processed for user `e938e615637f4c9291c60df5dfeb06ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:34.206476 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:34.244020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:34.335327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1426/2851] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:34.347322 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:34.349352 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:34.414470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a12d4-073b-43de-a0cb-9f0be8fd3142 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:34.419587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e25e30-1634-4185-86e2-41439fa28af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:34.420255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25e30-1634-4185-86e2-41439fa28af9 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:34.420616 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25e30-1634-4185-86e2-41439fa28af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:34.420961 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e25e30-1634-4185-86e2-41439fa28af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:34.434770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25e30-1634-4185-86e2-41439fa28af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:34.436272 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e25e30-1634-4185-86e2-41439fa28af9 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:34.448799 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1426/2852] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:34.602453 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:34.676433 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f35a-f19a-4de4-9613-5bd7568f58f7 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:34.682617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:34.683454 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:34.683971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:34.683971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] PATH_INFO: `/v3/users/c77f7154c18c4151b39b9a63e90f1189` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:34.685424 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77f7154c18c4151b39b9a63e90f1189)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:34.687700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:34.748212 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77f7154c18c4151b39b9a63e90f1189'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:34.755402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77f7154c18c4151b39b9a63e90f1189'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:34.763157 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77f7154c18c4151b39b9a63e90f1189'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:34.789688 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77f7154c18c4151b39b9a63e90f1189'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:34.790515 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77f7154c18c4151b39b9a63e90f1189'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:34.797300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1427/2853] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:34 2022] DELETE /identity/v3/users/c77f7154c18c4151b39b9a63e90f1189 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:14:34.801535 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:34.801893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:34.802175 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:34.822686 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:34.824862 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:34.893786 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1427/2854] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:34.905891 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:34.908164 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:34.974030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:35.006912 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d968cbc-2eae-48a5-af25-695791162fc7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:35.012681 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:35.013452 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:35.013735 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:35.014020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:35.024351 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:35.026805 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:35.037885 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1428/2855] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:35.360222 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:35.361966 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:35.374970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5a6b23-af24-4700-8f4d-99bc811d0984 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:35.380835 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:35.381831 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:35.382215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:35.382587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] PATH_INFO: `/v3/projects/664876d2894c48ea8745c0b89ce2d23d` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:35.384046 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664876d2894c48ea8745c0b89ce2d23d)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:35.386167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:35.436735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:35.445980 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664876d2894c48ea8745c0b89ce2d23d', 'request_id': 'req-81706d47-6efe-4fc6-932a-1248db74d348'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.455840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664876d2894c48ea8745c0b89ce2d23d', 'request_id': 'req-81706d47-6efe-4fc6-932a-1248db74d348'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.467891 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664876d2894c48ea8745c0b89ce2d23d', 'request_id': 'req-81706d47-6efe-4fc6-932a-1248db74d348'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.486218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] The token cache is being invalidate because project 664876d2894c48ea8745c0b89ce2d23d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:35.486475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.488420 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.490467 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1428/2856] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:35 2022] DELETE /identity/v3/projects/664876d2894c48ea8745c0b89ce2d23d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:35.496536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:35.508371 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf2a5-9429-4245-b9da-258e78d570d7 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:35.586161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:35.586877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:35.587227 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:35.587567 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] PATH_INFO: `/v3/users/e938e615637f4c9291c60df5dfeb06ea` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:35.589819 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e938e615637f4c9291c60df5dfeb06ea)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:35.591951 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:35.667349 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e938e615637f4c9291c60df5dfeb06ea'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.698905 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e938e615637f4c9291c60df5dfeb06ea'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.718247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e938e615637f4c9291c60df5dfeb06ea'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.746657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e938e615637f4c9291c60df5dfeb06ea'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.749059 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-38a1af8b-d989-4fc0-b577-4aba86d0fadc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e938e615637f4c9291c60df5dfeb06ea'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:35.758528 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1429/2857] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:35 2022] DELETE /identity/v3/users/e938e615637f4c9291c60df5dfeb06ea => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:14:36.245422 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:36.268691 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:36.299517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d47-6efe-4fc6-932a-1248db74d348 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:36.402478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:36.403636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.404661 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.405403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] PATH_INFO: `/v3/projects/231ba747c0474f6198d937c293e196f4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.408060 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=231ba747c0474f6198d937c293e196f4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:36.412350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:36.489584 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '231ba747c0474f6198d937c293e196f4', 'request_id': 'req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:36.503613 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '231ba747c0474f6198d937c293e196f4', 'request_id': 'req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:36.512836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '231ba747c0474f6198d937c293e196f4', 'request_id': 'req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:36.528066 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] The token cache is being invalidate because project 231ba747c0474f6198d937c293e196f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:36.528351 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:36.529061 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6dd119de-bc09-4f13-bcde-746bbe2ccfb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:36.530617 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1429/2858] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:36 2022] DELETE /identity/v3/projects/231ba747c0474f6198d937c293e196f4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:36.535907 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.536231 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.536511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.560926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:36.562199 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:36.621661 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:36.633625 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1430/2859] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:36.641689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.641975 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.642255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.663113 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:36.664404 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:36.674186 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1430/2860] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:36.678402 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:36.689758 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2339463-e967-4d99-adf0-455ca9a6a07a None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:36.694604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:36.695233 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.695512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.695787 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.696506 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:14:36.696809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:36.698687 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:36.711623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1431/2861] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:14:36.717812 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:36.738370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad26b4aa-5ac9-4f23-a0a4-91f3c4cdb2a3 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:36.743231 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:36.743845 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.744236 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.744511 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.745156 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:36.747057 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:36.783867 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1431/2862] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:36 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:36.787146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:36.799975 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f021c75-9b95-437d-8ae4-9f3b19d89efe admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:36.874535 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:36.875482 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.876201 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.876521 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.877700 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:36.881979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:36.912109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1432/2863] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:36 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:36.916158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:36.928512 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329001a-0c6c-4957-87e6-ad6462cfe437 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:36.933453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:36.934056 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.934333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.934606 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.935252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:36.936911 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:36.947514 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1432/2864] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:36.952161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:36.963784 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8642792-6381-47df-9f29-ae62eaf70f0f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:36.968541 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aedcb-eab0-4479-accd-42c70ec89799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:36.969151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aedcb-eab0-4479-accd-42c70ec89799 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:36.969430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aedcb-eab0-4479-accd-42c70ec89799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:36.969701 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aedcb-eab0-4479-accd-42c70ec89799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:36.970353 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aedcb-eab0-4479-accd-42c70ec89799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:36.971969 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aedcb-eab0-4479-accd-42c70ec89799 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:36.984298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1433/2865] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:36.989044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.000336 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b367f36-6913-47bf-8873-01ad742c3dea admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.004944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.005647 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.005923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.006194 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] PATH_INFO: `/v3/projects/6db3499e808b4cd1863d6e69b87be807/users/45093a24ce0f496f97f61f528aab81b4/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.022312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db3499e808b4cd1863d6e69b87be807, user_id=45093a24ce0f496f97f61f528aab81b4, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.025181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.028441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:37.042520 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1433/2866] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:36 2022] PUT /identity/v3/projects/6db3499e808b4cd1863d6e69b87be807/users/45093a24ce0f496f97f61f528aab81b4/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:37.054270 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.054556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.054835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.085388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] MFA Rules not processed for user `45093a24ce0f496f97f61f528aab81b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:37.086623 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:37.156145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1434/2867] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:37.160652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.223232 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea70cfb-be63-49f9-924f-b67236dc2525 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.236836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.237458 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.237734 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.238003 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.238660 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.240607 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.268970 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1434/2868] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:37 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:37.273151 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.291149 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fea609-b751-4a98-87f0-559f33f81ef4 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.342244 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:37.351451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.352289 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.352614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.352960 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.354444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.356677 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.380156 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1435/2869] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:37 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:14:37.384462 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.395651 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06ca1b-5f97-4d3b-abbb-46140ead31ad admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.401020 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.401843 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.402190 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.402388 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.403177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.404699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.416857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1435/2870] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:37.420924 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.431419 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f238d5-675a-45e6-8b94-9f2a419afbe7 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.437099 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade676ab-576d-4885-910e-1a5da5e71fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.437723 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade676ab-576d-4885-910e-1a5da5e71fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.438004 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade676ab-576d-4885-910e-1a5da5e71fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.438273 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade676ab-576d-4885-910e-1a5da5e71fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.438926 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade676ab-576d-4885-910e-1a5da5e71fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.440565 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade676ab-576d-4885-910e-1a5da5e71fd0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.451308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1436/2871] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:37.455713 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.466909 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c477537a-3193-4a11-9bc8-8ec5ab73b265 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.471764 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.472490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.472771 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.473050 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] PATH_INFO: `/v3/projects/a248c52cb5b9460abca0e25e56b05fa4/users/ee7905cea5744a4ca21fb75015d70162/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.489818 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a248c52cb5b9460abca0e25e56b05fa4, user_id=ee7905cea5744a4ca21fb75015d70162, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.492548 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.508412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1436/2872] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:37 2022] PUT /identity/v3/projects/a248c52cb5b9460abca0e25e56b05fa4/users/ee7905cea5744a4ca21fb75015d70162/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:37.577748 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.578034 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.578322 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.610608 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] MFA Rules not processed for user `ee7905cea5744a4ca21fb75015d70162`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:37.611872 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:37.681261 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1437/2873] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:37.686459 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.751648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17224a79-4925-46db-b359-5c157c46fb61 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.756350 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.756959 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.757238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.757513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.758165 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.760144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.794963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1437/2874] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:37 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:14:37.799807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.811620 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc15c7-57cb-4133-b63f-2fdce5bc2861 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.878779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.879421 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.879709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.879986 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.880713 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.882734 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.910578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1438/2875] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:37 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:37.915117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.926557 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a516cc2-b018-4bf1-8aae-b1f76cfe60e5 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.932803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.933685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.933973 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.934254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.934969 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.936685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:37.951337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1438/2876] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:37.956505 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:37.967732 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cef048e-16c7-4e36-9d59-5ae416fde3ca admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:37.973098 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:37.973881 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:37.974253 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:37.974619 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] PATH_INFO: `/v3/projects/94854d1baf8b447eace021ee65b4eeb9/users/04539f0fe06d402fb7fa93df7de19a56/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:37.992564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94854d1baf8b447eace021ee65b4eeb9, user_id=04539f0fe06d402fb7fa93df7de19a56, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:37.995316 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:38.008997 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1439/2877] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:37 2022] PUT /identity/v3/projects/94854d1baf8b447eace021ee65b4eeb9/users/04539f0fe06d402fb7fa93df7de19a56/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:38.014218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:38.024735 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e176f7e6-3225-4029-91c9-b9710247577e admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:38.042710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:38.086670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4aacf4-0d94-4e8a-bf92-faa7cb97bfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:38.087277 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aacf4-0d94-4e8a-bf92-faa7cb97bfed admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:38.087559 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aacf4-0d94-4e8a-bf92-faa7cb97bfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:38.087836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4aacf4-0d94-4e8a-bf92-faa7cb97bfed admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:38.089397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aacf4-0d94-4e8a-bf92-faa7cb97bfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:38.092482 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4aacf4-0d94-4e8a-bf92-faa7cb97bfed admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:38.104366 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1439/2878] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:38.110298 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:38.121646 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bb083-79ac-4c50-a63d-36b1bd0bf2d4 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:38.126300 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:38.126948 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] REQUEST_METHOD: `PUT` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:38.127228 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:38.127502 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] PATH_INFO: `/v3/projects/94854d1baf8b447eace021ee65b4eeb9/users/04539f0fe06d402fb7fa93df7de19a56/roles/e6bfcb3b509648a0888a6485214b8ccb` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:38.129461 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=94854d1baf8b447eace021ee65b4eeb9, user_id=04539f0fe06d402fb7fa93df7de19a56, role_id=e6bfcb3b509648a0888a6485214b8ccb)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:38.132073 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:38.142488 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1440/2879] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:38 2022] PUT /identity/v3/projects/94854d1baf8b447eace021ee65b4eeb9/users/04539f0fe06d402fb7fa93df7de19a56/roles/e6bfcb3b509648a0888a6485214b8ccb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:38.153032 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27950147-8471-455f-91c4-845ca398a7db None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:38.153321 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27950147-8471-455f-91c4-845ca398a7db None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:38.153599 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27950147-8471-455f-91c4-845ca398a7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:38.181549 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-27950147-8471-455f-91c4-845ca398a7db None None] MFA Rules not processed for user `04539f0fe06d402fb7fa93df7de19a56`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:38.183636 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-27950147-8471-455f-91c4-845ca398a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:38.251754 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1440/2880] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:38 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:38.263712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:38.265444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:38.284989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:38.328448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94247d12-2134-4b37-a01f-ac0d2bf26caf admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:38.332080 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:38.332686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:38.332964 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:38.333236 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:38.343304 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:38.344774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:38.354917 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1441/2881] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:38 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:38.395417 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27950147-8471-455f-91c4-845ca398a7db None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:38.396840 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-27950147-8471-455f-91c4-845ca398a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:38.410984 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27950147-8471-455f-91c4-845ca398a7db None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:38.415051 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98e18bc-666d-4a47-9528-498d4fbee97d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:38.415643 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e18bc-666d-4a47-9528-498d4fbee97d service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:38.415923 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e18bc-666d-4a47-9528-498d4fbee97d service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:38.416864 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98e18bc-666d-4a47-9528-498d4fbee97d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:38.473002 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e18bc-666d-4a47-9528-498d4fbee97d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:38.474428 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98e18bc-666d-4a47-9528-498d4fbee97d service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:38.483501 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1441/2882] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:41.401682 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:41.469696 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83fedc-40ab-4ff2-b89b-a1b44c838f58 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:41.475039 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:41.475666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:41.475940 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:41.476238 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] PATH_INFO: `/v3/users/45093a24ce0f496f97f61f528aab81b4` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:41.477390 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45093a24ce0f496f97f61f528aab81b4)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:41.479898 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:41.530598 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45093a24ce0f496f97f61f528aab81b4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:41.537880 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45093a24ce0f496f97f61f528aab81b4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:41.546971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45093a24ce0f496f97f61f528aab81b4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:41.560783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45093a24ce0f496f97f61f528aab81b4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:41.561422 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45093a24ce0f496f97f61f528aab81b4'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:41.567807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1442/2883] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:41 2022] DELETE /identity/v3/users/45093a24ce0f496f97f61f528aab81b4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:41.573135 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:41.573664 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:41.573803 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:41.593202 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:41.594430 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:41.631652 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:41.659877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1442/2884] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:41.670318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:41.671768 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:41.715919 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:41.734074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd83f7b-a3dd-4a8d-b213-824cd42bcc10 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:41.739607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:41.740327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:41.740634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:41.740936 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:41.751451 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:41.753138 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:41.764358 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1443/2885] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:14:42.049596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:42.051018 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:42.064254 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9f13a-7c5d-4bfb-abb5-77983bf1ea03 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:42.069477 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:42.070133 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:42.070442 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:42.070719 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] PATH_INFO: `/v3/projects/6db3499e808b4cd1863d6e69b87be807` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:42.073317 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db3499e808b4cd1863d6e69b87be807)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:42.075635 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:42.126158 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db3499e808b4cd1863d6e69b87be807', 'request_id': 'req-6ba4f7b1-8e75-4835-a79f-098a875ff156'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.135346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db3499e808b4cd1863d6e69b87be807', 'request_id': 'req-6ba4f7b1-8e75-4835-a79f-098a875ff156'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.141974 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db3499e808b4cd1863d6e69b87be807', 'request_id': 'req-6ba4f7b1-8e75-4835-a79f-098a875ff156'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.153922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] The token cache is being invalidate because project 6db3499e808b4cd1863d6e69b87be807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:42.154215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.154880 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.156403 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1443/2886] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:42 2022] DELETE /identity/v3/projects/6db3499e808b4cd1863d6e69b87be807 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:42.160485 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:42.171178 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90153-eabe-43ae-9ea3-c83387330e3f service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:42.241899 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:42.242536 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:42.242814 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:42.243089 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] PATH_INFO: `/v3/users/ee7905cea5744a4ca21fb75015d70162` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:42.245113 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee7905cea5744a4ca21fb75015d70162)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:42.247074 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:42.298746 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee7905cea5744a4ca21fb75015d70162'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.305747 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee7905cea5744a4ca21fb75015d70162'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.314448 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee7905cea5744a4ca21fb75015d70162'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.329221 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee7905cea5744a4ca21fb75015d70162'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.329856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee7905cea5744a4ca21fb75015d70162'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.335945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1444/2887] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:42 2022] DELETE /identity/v3/users/ee7905cea5744a4ca21fb75015d70162 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:42.613591 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:42.616019 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:42.637720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba4f7b1-8e75-4835-a79f-098a875ff156 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:42.698324 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:42.698968 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:42.699244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:42.699518 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] PATH_INFO: `/v3/projects/a248c52cb5b9460abca0e25e56b05fa4` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:42.700696 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a248c52cb5b9460abca0e25e56b05fa4)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:42.702603 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:42.753239 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a248c52cb5b9460abca0e25e56b05fa4', 'request_id': 'req-544abce4-5b54-40cc-9b4b-212dc9c70c9a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.762274 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a248c52cb5b9460abca0e25e56b05fa4', 'request_id': 'req-544abce4-5b54-40cc-9b4b-212dc9c70c9a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.768359 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a248c52cb5b9460abca0e25e56b05fa4', 'request_id': 'req-544abce4-5b54-40cc-9b4b-212dc9c70c9a'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.769465 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:42.789495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] The token cache is being invalidate because project a248c52cb5b9460abca0e25e56b05fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:42.789972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.791213 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.793176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1444/2888] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:42 2022] DELETE /identity/v3/projects/a248c52cb5b9460abca0e25e56b05fa4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:42.797698 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:42.799172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:42.812695 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74211618-05d2-4674-a5dc-e13ee533818b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:42.872246 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:42.872791 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:42.873069 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:42.873344 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] PATH_INFO: `/v3/users/04539f0fe06d402fb7fa93df7de19a56` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:42.874607 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04539f0fe06d402fb7fa93df7de19a56)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:42.876582 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:42.878146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:42.928669 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04539f0fe06d402fb7fa93df7de19a56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.936269 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04539f0fe06d402fb7fa93df7de19a56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.944075 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04539f0fe06d402fb7fa93df7de19a56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.957046 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04539f0fe06d402fb7fa93df7de19a56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.957745 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-1fdb892a-0a24-452a-bd8a-f8ae1d543431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04539f0fe06d402fb7fa93df7de19a56'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:42.963756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1445/2889] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:14:42 2022] DELETE /identity/v3/users/04539f0fe06d402fb7fa93df7de19a56 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:14:43.264756 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.268423 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:43.285476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544abce4-5b54-40cc-9b4b-212dc9c70c9a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.347284 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.347918 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.348225 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.348495 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] PATH_INFO: `/v3/projects/94854d1baf8b447eace021ee65b4eeb9` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.349630 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94854d1baf8b447eace021ee65b4eeb9)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.351523 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.400832 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94854d1baf8b447eace021ee65b4eeb9', 'request_id': 'req-27082521-46e6-4229-8503-078ff976f6a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:43.408957 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94854d1baf8b447eace021ee65b4eeb9', 'request_id': 'req-27082521-46e6-4229-8503-078ff976f6a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:43.414770 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94854d1baf8b447eace021ee65b4eeb9', 'request_id': 'req-27082521-46e6-4229-8503-078ff976f6a1'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:43.426827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] The token cache is being invalidate because project 94854d1baf8b447eace021ee65b4eeb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:14:43.427116 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:43.427814 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-27082521-46e6-4229-8503-078ff976f6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:14:43.429363 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1445/2890] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:14:43 2022] DELETE /identity/v3/projects/94854d1baf8b447eace021ee65b4eeb9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:43.436360 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.436649 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.436930 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.457200 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:43.458416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:43.550657 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1446/2891] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:43.562180 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.562760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.563029 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.584261 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:43.585478 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:43.595976 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1446/2892] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:43.601773 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.612666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3670f-79ef-4aae-bf8d-369eee476ab3 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.617471 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.618102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.618386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.618662 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.619416 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:14:43.619961 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.621779 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.636459 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1447/2893] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:14:43.641103 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.651733 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ab38c-22c0-4d16-8957-ced488f723be None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.656592 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.657205 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.657490 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.657762 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.658409 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.660346 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.691545 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1447/2894] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:14:43 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:43.697185 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.709015 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a3368-6b8e-488e-a439-4f507f2e577b admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.769577 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.770181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.770475 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.770729 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.771366 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.773305 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.786835 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:43.805193 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1448/2895] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:14:43 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 11:14:43.810177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.820873 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a53862-8590-4e99-923a-412f672dc488 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.827196 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.827806 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.828115 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.828389 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.829036 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.830807 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.842626 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1448/2896] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:43.846985 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.857446 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51aa56-69ce-40ef-866a-6cd0c8c8c8e3 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.862361 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee94917-e7c9-4de7-8ef2-733ada3d9612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.862989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94917-e7c9-4de7-8ef2-733ada3d9612 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.863266 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94917-e7c9-4de7-8ef2-733ada3d9612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.863534 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee94917-e7c9-4de7-8ef2-733ada3d9612 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.864224 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee94917-e7c9-4de7-8ef2-733ada3d9612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.865824 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee94917-e7c9-4de7-8ef2-733ada3d9612 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.877744 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1449/2897] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:14:43.882851 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:43.894525 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c0de6-0703-4253-bc44-c3805dabf8ff admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:43.899928 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:43.902415 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.902765 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.903146 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] PATH_INFO: `/v3/projects/35edc48197524087862c5cbf77f52fca/users/916fde6d9347426d84a539e44e65525f/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:43.920637 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35edc48197524087862c5cbf77f52fca, user_id=916fde6d9347426d84a539e44e65525f, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:43.923905 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:43.944117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1449/2898] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:14:43 2022] PUT /identity/v3/projects/35edc48197524087862c5cbf77f52fca/users/916fde6d9347426d84a539e44e65525f/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:14:43.957437 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:43.957945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:43.958439 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:44.003152 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] MFA Rules not processed for user `916fde6d9347426d84a539e44e65525f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:14:44.004435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:44.075866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1450/2899] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:14:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:14:44.090382 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:14:44.092155 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:14:44.117044 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:44.161801 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921fbff-b494-417c-a6c3-63f4f9ac72d2 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:14:44.165529 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4002cc3-5505-4f1f-b63a-39e2b55c75db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:14:44.166745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4002cc3-5505-4f1f-b63a-39e2b55c75db service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:14:44.167045 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4002cc3-5505-4f1f-b63a-39e2b55c75db service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:14:44.167370 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4002cc3-5505-4f1f-b63a-39e2b55c75db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:14:44.176314 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4002cc3-5505-4f1f-b63a-39e2b55c75db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:14:44.177744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4002cc3-5505-4f1f-b63a-39e2b55c75db service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:14:44.187857 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1450/2900] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:14:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:14:54.073496 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=39 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:14:54.183412 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=17 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:14.476932 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:14.557764 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97be5f-3073-4e8d-a19c-44a02264001f None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:14.562601 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:14.563247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:14.563531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:14.563810 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] PATH_INFO: `/v3/users/916fde6d9347426d84a539e44e65525f` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:14.565047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=916fde6d9347426d84a539e44e65525f)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:14.566989 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:14.621503 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916fde6d9347426d84a539e44e65525f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:14.630318 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916fde6d9347426d84a539e44e65525f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:14.640321 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916fde6d9347426d84a539e44e65525f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:14.654842 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916fde6d9347426d84a539e44e65525f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:14.655578 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916fde6d9347426d84a539e44e65525f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:14.662187 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1451/2901] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:15:14 2022] DELETE /identity/v3/users/916fde6d9347426d84a539e44e65525f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:15:14.668312 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:14.668927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:14.668927 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:14.688505 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:14.689774 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:14.758207 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1451/2902] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:15:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:15:14.769597 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:14.771176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:14.841047 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6a370-5242-40d9-97a9-0bacc6e72e92 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:14.844914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a9895-8e52-476b-a51b-6350944f21d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:14.845531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9895-8e52-476b-a51b-6350944f21d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:14.845813 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9895-8e52-476b-a51b-6350944f21d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:14.846090 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a9895-8e52-476b-a51b-6350944f21d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:14.857252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a9895-8e52-476b-a51b-6350944f21d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:14.858726 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a9895-8e52-476b-a51b-6350944f21d3 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:14.870455 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1452/2903] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:15:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:15:15.186042 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.187819 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:15.202083 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278af95a-7639-4d6f-bece-eea5b749970a None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.209154 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.209863 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.210244 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.210615 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] PATH_INFO: `/v3/projects/35edc48197524087862c5cbf77f52fca` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.213433 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=35edc48197524087862c5cbf77f52fca)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.215800 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.277704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35edc48197524087862c5cbf77f52fca', 'request_id': 'req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:15.286648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35edc48197524087862c5cbf77f52fca', 'request_id': 'req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:15.293227 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35edc48197524087862c5cbf77f52fca', 'request_id': 'req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:15.306081 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] The token cache is being invalidate because project 35edc48197524087862c5cbf77f52fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81463) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:15:15.306587 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:15.307457 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-09768af2-5f79-4eb6-bc3a-ac1b28d592ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:15:15.310086 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1452/2904] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:15:15 2022] DELETE /identity/v3/projects/35edc48197524087862c5cbf77f52fca => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:15:15.317276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.317614 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.318017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.339710 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:15.341022 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:15.360616 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:15.411740 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1453/2905] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:15:15.424624 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.424913 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.425197 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.448214 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:15.449487 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:15.457836 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1453/2906] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 11:15:15.462604 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.473160 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715d913-4b7f-42a6-a0d5-a244d29671c5 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.478783 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.479804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.480335 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.480809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.481944 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81462) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 11:15:15.482564 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.484556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.498686 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1454/2907] 10.222.0.20 () {58 vars in 1153 bytes} [Wed Aug 31 11:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 11:15:15.505176 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.515655 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96711d-7d39-46b3-a195-f7df6f3a05bc None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.520821 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.521576 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.521954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.522337 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] PATH_INFO: `/v3/projects` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.523444 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.526298 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.565491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1454/2908] 10.222.0.20 () {60 vars in 1153 bytes} [Wed Aug 31 11:15:15 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:15:15.570247 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.582774 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427a03b-e6d3-4c63-a9f6-d97193925b4a admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.645856 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.646495 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.646777 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.647053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] PATH_INFO: `/v3/users` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.647971 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.650522 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.673062 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1455/2909] 10.222.0.20 () {60 vars in 1144 bytes} [Wed Aug 31 11:15:15 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 11:15:15.677685 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.688912 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815dd2a-433c-49fd-92b4-0bcbba0fd80a admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.693999 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.694610 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.694893 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.695191 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.695838 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.698342 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.709484 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1455/2910] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 11:15:15.714223 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.724644 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b299d1-a431-4f0f-afd3-1512bcd8bbdf admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.729531 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ee3e6-5ad6-4283-b66e-3fff39b5d023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.730146 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ee3e6-5ad6-4283-b66e-3fff39b5d023 admin admin] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.730430 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ee3e6-5ad6-4283-b66e-3fff39b5d023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.730705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ee3e6-5ad6-4283-b66e-3fff39b5d023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.731388 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ee3e6-5ad6-4283-b66e-3fff39b5d023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.733066 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ee3e6-5ad6-4283-b66e-3fff39b5d023 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.746756 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1456/2911] 10.222.0.20 () {58 vars in 1122 bytes} [Wed Aug 31 11:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 11:15:15.752085 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.763810 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda58472-e9f1-4dbe-9bde-23a90d35d0a1 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.768877 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.769558 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] REQUEST_METHOD: `PUT` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.769868 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.770147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] PATH_INFO: `/v3/projects/1fc761089b5740ecb88c71ff97f3f270/users/28aff6cc28ab4118bb334d7249697e9f/roles/8249b61571f3472089a8d3cc13b0c9df` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.787441 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fc761089b5740ecb88c71ff97f3f270, user_id=28aff6cc28ab4118bb334d7249697e9f, role_id=8249b61571f3472089a8d3cc13b0c9df)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:15.790285 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:15.806659 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1456/2912] 10.222.0.20 () {60 vars in 1483 bytes} [Wed Aug 31 11:15:15 2022] PUT /identity/v3/projects/1fc761089b5740ecb88c71ff97f3f270/users/28aff6cc28ab4118bb334d7249697e9f/roles/8249b61571f3472089a8d3cc13b0c9df => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:15:15.812705 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d13aa-1b17-4a1a-8068-44cdded05e34 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.812981 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d13aa-1b17-4a1a-8068-44cdded05e34 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.813252 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4d13aa-1b17-4a1a-8068-44cdded05e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:15.834172 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-4a4d13aa-1b17-4a1a-8068-44cdded05e34 None None] MFA Rules not processed for user `a29c05a14dda47f2a0f073398ac0dadb`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:15.835309 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-4a4d13aa-1b17-4a1a-8068-44cdded05e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:15.904886 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1457/2913] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:15:15.916132 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:15.917964 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:15.932560 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:15.987922 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eddf1f-74fb-40bd-967f-c78a83ca4724 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:15.992453 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:15.993767 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:15.994096 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:15.994596 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:16.005983 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:16.007710 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:16.020917 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1457/2914] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:15:17.177176 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237c131-c964-401a-86b9-718ab5801a68 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:17.177807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237c131-c964-401a-86b9-718ab5801a68 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:17.177807 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237c131-c964-401a-86b9-718ab5801a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:17.211092 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-7237c131-c964-401a-86b9-718ab5801a68 None None] MFA Rules not processed for user `28aff6cc28ab4118bb334d7249697e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:17.213056 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-7237c131-c964-401a-86b9-718ab5801a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:17.238319 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:17.296692 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1458/2915] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:15:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:15:17.311395 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:17.313725 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:17.401553 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9065-b492-4480-903e-8ad324aee163 service neutron] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:17.405962 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:17.406614 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:17.406897 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:17.407255 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:17.426181 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:17.427707 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:17.438534 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1458/2916] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:15:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:15:18.848035 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] REQUEST_METHOD: `POST` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:18.848609 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:18.849581 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:18.870916 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.auth.core [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] MFA Rules not processed for user `28aff6cc28ab4118bb334d7249697e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81462) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:15:18.872192 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:18.879145 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1459/2917] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:15:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:15:18.920473 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:18.921944 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:18.982151 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e2b88-d63f-4691-a328-592c90fb1681 service cinder] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:18.986144 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72251fea0df54206b554236c14296a7b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:18.986748 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:18.987022 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:18.987291 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:18.989410 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:19.068915 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:19.070279 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:19.075517 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=100 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:19.085237 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1459/2918] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:15:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:15:21.917994 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:15:21.920053 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:21.936719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37122731-5c61-49b0-b5bf-60a755268c56 None None] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:15:21.940546 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:15:21.941147 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:15:21.941463 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:15:21.941693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:15:21.943665 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:15:21.958808 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:15:21.961013 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:15:21.973308 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1460/2919] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:15:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:15:29.086215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:15:31.967195 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=28 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:16:10.242203 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:10.244091 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:10.260411 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2c7e5-7f7c-4ac5-97a0-8be1b0f1c0d7 service glance] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:10.267437 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:10.268503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:10.270820 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:10.271885 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:10.274760 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:10.293215 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:10.297127 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c680f1-79e7-4efb-87c8-8af087872eb1 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:10.318409 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1460/2920] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:16:19.471412 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:19.472968 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:19.486007 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3449e336-0db2-4d3d-8ac4-656fdf05bdfd service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:19.491050 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1aaf56b3e654c0bb65d0c804fd797e6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:19.491785 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:19.492393 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:19.492709 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:19.494693 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:19.578017 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:19.579523 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:19.591339 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1461/2921] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:16:19.843184 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:19.845470 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:19.846177 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:19.874333 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] MFA Rules not processed for user `28aff6cc28ab4118bb334d7249697e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:16:19.875716 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:19.883369 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1461/2922] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:16:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:16:20.180760 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:20.191520 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d35a8-429e-4fa0-8696-2383ce4065c6 service cinder] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:20.195512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:20.196165 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:20.196444 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:20.196714 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:20.199315 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:20.218809 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:20.221666 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:20.238556 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1462/2923] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:16:20.669910 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:20.671381 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:20.766753 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f7ccb-83d9-4d18-b538-7352a8ec9b5c None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:20.777052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:20.777052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:20.777052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:20.777052 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:20.777797 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:20.789829 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:20.791492 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da48877-3ccb-4bb3-93ec-1e69b7c60d3f service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:20.804171 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1462/2924] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:16:24.771554 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:24.785209 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f411d60-826a-42a8-9868-e0a82a442ed1 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:24.791937 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:24.793014 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:24.793512 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:24.793998 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:24.797435 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:24.818847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:24.821755 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:24.838561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1463/2925] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:16:25.894670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] REQUEST_METHOD: `POST` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:25.895530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:25.895717 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:25.931906 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.auth.core [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] MFA Rules not processed for user `16696c34de254f3babf538c879f2ce49`. Rule list: `[]` (Enabled: `True`). {{(pid=81463) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 11:16:25.934201 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:25.951489 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1463/2926] 10.222.0.20 () {58 vars in 958 bytes} [Wed Aug 31 11:16:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 11:16:26.846181 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:26.861876 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d690fcc-8462-471d-91e6-4f44ee574353 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:26.869757 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:26.870786 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:26.871530 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:26.871801 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:26.877217 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:26.898347 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:26.903005 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:26.918634 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1464/2927] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:16:32.350530 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:16:32.382498 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fd0c4a-b95d-4d22-b596-b583e84a2c59 None None] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:16:32.395597 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '508f557ac5e34c67a31e4d55f8142a0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:16:32.396475 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:16:32.396856 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:16:32.397252 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:16:32.399675 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:16:32.416745 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:16:32.418577 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:16:32.433513 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1464/2928] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:16:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:16:36.912123 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=46 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:16:42.426436 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=50 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:17:21.610708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:17:21.624561 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8cfbd-d122-4599-911f-1c6d5e876679 service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:17:21.631276 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:17:21.631909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:17:21.632225 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:17:21.632509 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:17:21.634386 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:17:21.649020 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:17:21.650845 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:17:21.666281 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1465/2929] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:17:21.697744 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:17:21.709699 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963bceb-78b3-45c1-8e0e-f704e97b04a4 service nova] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:17:21.715179 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:17:21.716193 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:17:21.716648 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:17:21.717138 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] PATH_INFO: `/v3/users/28aff6cc28ab4118bb334d7249697e9f` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:17:21.718704 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28aff6cc28ab4118bb334d7249697e9f)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:17:21.721340 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:17:21.795570 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28aff6cc28ab4118bb334d7249697e9f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:21.796720 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28aff6cc28ab4118bb334d7249697e9f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:21.807476 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28aff6cc28ab4118bb334d7249697e9f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:21.823330 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28aff6cc28ab4118bb334d7249697e9f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:21.831262 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28aff6cc28ab4118bb334d7249697e9f'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:21.837705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1465/2930] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:17:21 2022] DELETE /identity/v3/users/28aff6cc28ab4118bb334d7249697e9f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:17:22.039109 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:17:22.056133 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d485d7-1ea9-4b22-ae5d-f6b8022eaed6 service placement] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:17:22.120877 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:17:22.121638 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:17:22.121949 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:17:22.122275 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] PATH_INFO: `/v3/projects/1fc761089b5740ecb88c71ff97f3f270` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:17:22.123501 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fc761089b5740ecb88c71ff97f3f270)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:17:22.125715 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:17:22.179183 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fc761089b5740ecb88c71ff97f3f270', 'request_id': 'req-b16f128b-f847-461b-9b8e-443b62131a3f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:22.186137 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fc761089b5740ecb88c71ff97f3f270', 'request_id': 'req-b16f128b-f847-461b-9b8e-443b62131a3f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:22.194499 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fc761089b5740ecb88c71ff97f3f270', 'request_id': 'req-b16f128b-f847-461b-9b8e-443b62131a3f'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:22.209617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] The token cache is being invalidate because project 1fc761089b5740ecb88c71ff97f3f270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:17:22.209909 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:22.210473 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:17:22.212030 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1466/2931] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:17:22 2022] DELETE /identity/v3/projects/1fc761089b5740ecb88c71ff97f3f270 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:17:31.834167 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:17:32.206000 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:18:33.046900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:33.049087 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:33.145854 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff17cfa-af92-4b40-be4d-f8ce7ff6ec66 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:33.238958 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f5103a11f8f446778043577e13cea822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:33.239837 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] REQUEST_METHOD: `GET` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:33.240372 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:33.240963 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:33.243705 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:33.263933 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:33.266972 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:33.288706 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1466/2932] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:18:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 11:18:43.280516 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=58 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:18:44.003338 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:44.005391 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:44.076803 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f128b-f847-461b-9b8e-443b62131a3f admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:44.080781 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49f7536740a54bdbba5b6c81f732c439', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3a0a65f6805f4a5996e5fccef0bcf57e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:44.081481 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] REQUEST_METHOD: `GET` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:44.081847 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:44.082161 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:44.084194 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:44.158712 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:44.160511 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:44.174324 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1467/2933] 10.222.0.20 () {60 vars in 1365 bytes} [Wed Aug 31 11:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 11:18:52.505556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:52.507306 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:52.524153 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ea5eb-324e-4326-a18e-349a3f6c64ee service placement] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:52.530007 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:52.530780 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:52.531117 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:52.531387 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] PATH_INFO: `/v3/users/16696c34de254f3babf538c879f2ce49` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:52.532622 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16696c34de254f3babf538c879f2ce49)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:52.534561 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:52.594556 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16696c34de254f3babf538c879f2ce49'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.595152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16696c34de254f3babf538c879f2ce49'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.604461 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16696c34de254f3babf538c879f2ce49'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.618218 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16696c34de254f3babf538c879f2ce49'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.626253 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16696c34de254f3babf538c879f2ce49'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.633031 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1467/2934] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:18:52 2022] DELETE /identity/v3/users/16696c34de254f3babf538c879f2ce49 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:18:52.837719 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:52.851425 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af2b32-9366-4f11-ab89-968f3d2b512b service neutron] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:52.925447 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:52.926102 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:52.926382 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:52.926650 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] PATH_INFO: `/v3/projects/da5b36cb688f49e1bf1cf75a108858fa` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:52.927860 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5b36cb688f49e1bf1cf75a108858fa)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:52.929823 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:52.982061 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5b36cb688f49e1bf1cf75a108858fa', 'request_id': 'req-b5265a8b-085a-4c5b-8eab-029d79654255'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.984708 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:18:52.989643 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5b36cb688f49e1bf1cf75a108858fa', 'request_id': 'req-b5265a8b-085a-4c5b-8eab-029d79654255'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:52.997327 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5b36cb688f49e1bf1cf75a108858fa', 'request_id': 'req-b5265a8b-085a-4c5b-8eab-029d79654255'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.010945 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] The token cache is being invalidate because project da5b36cb688f49e1bf1cf75a108858fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:18:53.011235 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.012979 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.014804 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1468/2935] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:18:52 2022] DELETE /identity/v3/projects/da5b36cb688f49e1bf1cf75a108858fa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:18:53.019152 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Authenticating user token {{(pid=81463) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:53.021238 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.fernet_utils [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81463) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:53.036397 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb771ff-27b3-4fce-8643-6bd1b8d0c2f9 admin admin] Validating token access rules against request {{(pid=81463) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:53.099900 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81463) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:53.100689 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:53.101079 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:53.101491 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] PATH_INFO: `/v3/users/8a89943b6692461789bec2e224e845c6` {{(pid=81463) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:53.102890 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a89943b6692461789bec2e224e845c6)` {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:53.105035 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] RBAC: Authorization granted {{(pid=81463) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:53.161827 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a89943b6692461789bec2e224e845c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.162503 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a89943b6692461789bec2e224e845c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.172665 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a89943b6692461789bec2e224e845c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.187515 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a89943b6692461789bec2e224e845c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.189670 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:18:53.197954 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG keystone.notifications [None req-c72ac594-8935-4c53-931b-3bcffec628f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a89943b6692461789bec2e224e845c6'} {{(pid=81463) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.206145 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: [pid: 81463|app: 0|req: 1468/2936] 10.222.0.20 () {58 vars in 1224 bytes} [Wed Aug 31 11:18:53 2022] DELETE /identity/v3/users/8a89943b6692461789bec2e224e845c6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 11:18:53.406617 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Authenticating user token {{(pid=81462) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 11:18:53.408222 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.fernet_utils [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81462) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 11:18:53.424956 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265a8b-085a-4c5b-8eab-029d79654255 admin admin] Validating token access rules against request {{(pid=81462) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 11:18:53.493952 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a29c05a14dda47f2a0f073398ac0dadb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '34b2b6cdb81943ad9b9237874eeca93b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81462) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 11:18:53.494599 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 11:18:53.494878 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 11:18:53.495150 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] PATH_INFO: `/v3/projects/694830b09d0d4ed0be53202125eb5398` {{(pid=81462) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 11:18:53.496984 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=694830b09d0d4ed0be53202125eb5398)` {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 11:18:53.498914 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] RBAC: Authorization granted {{(pid=81462) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 11:18:53.558401 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '694830b09d0d4ed0be53202125eb5398', 'request_id': 'req-59971745-5343-481b-adbe-8bb065345ce0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.566957 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '694830b09d0d4ed0be53202125eb5398', 'request_id': 'req-59971745-5343-481b-adbe-8bb065345ce0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.574016 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '694830b09d0d4ed0be53202125eb5398', 'request_id': 'req-59971745-5343-481b-adbe-8bb065345ce0'} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.588591 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] The token cache is being invalidate because project 694830b09d0d4ed0be53202125eb5398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81462) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 11:18:53.588866 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.589549 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG keystone.notifications [None req-59971745-5343-481b-adbe-8bb065345ce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81462) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 11:18:53.592103 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: [pid: 81462|app: 0|req: 1469/2937] 10.222.0.20 () {58 vars in 1233 bytes} [Wed Aug 31 11:18:53 2022] DELETE /identity/v3/projects/694830b09d0d4ed0be53202125eb5398 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 11:19:03.202147 dv-cinder-855350-1-iscsi devstack@keystone.service[81463]: DEBUG dbcounter [-] [81463] Writing DB stats keystone:SELECT=4 {{(pid=81463) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 11:19:03.585132 dv-cinder-855350-1-iscsi devstack@keystone.service[81462]: DEBUG dbcounter [-] [81462] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81462) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}